Control device, picking system, distribution system, program, control method and production method

ABSTRACT

A trajectory information generating unit is included, the trajectory information generating unit generating trajectory information that defines a trajectory for which a picking hand picks one of one or more works stored in one supply container included in one or more supply containers and arranges the work in one arrangement container included in one or more arrangement containers. After performing a process of generating trajectory information representing a first work included in the one or more works and before a movement of the first work is completed, the trajectory information generating unit may generate trajectory information representing a second work included in the one or more work and different from the first work by using an operation result in the process of generating the trajectory information regarding the first work.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Patent Application No. PCT/JP2017/014147 filed on Apr. 4, 2017, the contents of which are hereby incorporated herein by reference in its entirety.

BACKGROUND Technical Field

The subject disclosure relates to a control device, a picking system, a distribution system, a program, a control method and a production method.

Background

A picking system has been known, in which a picking hand and a robot arm are used to take out a work stored in a container and arrange the work on a specific position within another container.

CITATION LIST Patent Literature

[PTL 1]

Japanese Patent Application Publication No. 2016-091053

[PTL 2]

Japanese Patent Application Publication No. 2016-147330

[PTL 3]

Japanese Patent Application Publication No. 2017-033429

SUMMARY

In a case of automatically calculating a trajectory of a work by taking an interference with an obstacle into account, there are numerous amounts of calculation and it is difficult to calculate a trajectory within practical time.

In a first aspect of the present invention, a control device is provided. The above-described control device may be a control device for controlling a picking apparatus. In above-described control device, the picking apparatus may move, by using a picking hand, one or more works from one or more supply containers to one or more arrangement containers. The above-described control device includes, for example, a trajectory information generating unit that generates trajectory information that prescribes a trajectory for which a picking hand picks one of one or more works stored in one supply container included in one or more supply containers and arranges the work in one arrangement container included in one or more arrangement containers. In above-described control device, after performing a process of generating a trajectory information representing a first work included in one or more works and before the movement of the first work is completed, the trajectory information generating unit may generate trajectory information representing a second work, included in the one or more works and different from the first work, by using an operation result in the process of generating the trajectory information representing the first work.

The above-described control device may include a container information acquiring unit that acquires information representing at least one of a state of a surface of one supply container and a state of a surface of one arrangement container. In the above-described control device, the trajectory information generating unit may (a) acquire or generate information regarding a position and a posture of the first work in one supply container before the picking apparatus moves the first work during the process of generating the trajectory information representing the first work. In the above-described control device, the trajectory information generating unit may (b) generate trajectory information representing the second work by using (i) information acquired by the container information acquiring unit before the movement of the first work is completed and (ii) information representing the position and the posture of the first work in one supply container during the process of generating the trajectory information representing the second work.

The above-described control device may include a container information acquiring unit that acquires information representing a state of a surface of at least one of one supply container and one arrangement container. In the above-described control device, the trajectory information generating unit may (a) acquire or generate the information representing the position and the posture of the first work in one arrangement container after the picking apparatus moves the first work during the process of generating the trajectory information representing the first work. In the above-described control device, the trajectory information generating unit may (b) generate the trajectory information representing the second work by using (i) information acquired by the container information acquiring unit before the movement of the first work is completed and (ii) information representing the position and the posture of the first work in one arrangement container during the process of generating the trajectory information representing the second work.

The above-described control device may include an order information acquiring unit that acquires (i) first order information that includes instructions to move one or more works from at least one of one or more supply containers to a first arrangement container included in one or more arrangement containers and (ii) second order information that includes instructions to move one or more works from at least one of one or more supply containers to a second arrangement container that is included in the one or more arrangement containers and different from the first arrangement container. In above-described control device, the trajectory information generating unit may perform process of generating the trajectory information representing the first work in accordance with one of the first order information and the second order information acquired by the order information acquiring unit. In the above-described control device, the trajectory information generating unit may perform the process of generating the trajectory information representing the second work in accordance with the other of the first order information and the second order information acquired by the order information acquiring unit.

In above-described control device, the first order information may instruct to move the first work from the first supply container included in the one or more supply containers to a first arrangement container. In the above-described control device, the second order information may instruct to move the second work from the first supply container to the second arrangement container. In the above-described control device, the trajectory information generating unit may perform the process of generating the trajectory information representing the first work in accordance with the first order information acquired by the order information acquiring unit. In above-described control device, the trajectory information generating unit may perform the process of generating the trajectory information representing the second work in accordance with the second order information acquired by the order information acquiring unit.

The above-described control device may include an order information acquiring unit that acquires (i) first order information that includes instructions to move one or more works from a first supply container included in one or more supply containers to at least one of one or more arrangement containers, and (ii) second order information that includes instructions to move one or more works from a second supply container, different from the first supply container and included in the one or more supply containers and different from the first supply container, to at least one of the one or more arrangement containers. In the above-described control device, the trajectory information generating unit may perform the process of generating the trajectory information representing the first work in accordance with one of the first order information and the second order information acquired by the order information acquiring unit. In the above-described control device, the trajectory information generating unit may perform the process of generating the trajectory information representing the second work in accordance with the other of the first order information and the second order information acquired by the order information acquiring unit.

In the above-described control device, the first order information may instruct to move the first work from the first supply container to the first arrangement container included in the one or more supply containers. In the above-described control device, the second order information may instruct to move the second work from the second supply container to the first arrangement container. In above-described control device, the trajectory information generating unit may perform the process of generating the trajectory information representing the first work in accordance with the first order information acquired by the order information acquiring unit. In above-described control device, the trajectory information generating unit may perform the process of generating the trajectory information representing the second work in accordance with the second order information acquired by the order information acquiring unit.

The above-described control device may include an order information acquiring unit that acquires (i) first order information that includes instructions to move the first work from the first supply container included in the one or more supply containers to the first arrangement container included in the one or more arrangement containers and (ii) second order information that includes instructions to move the second work from the second supply container, different from the first supply container and included in the one or more supply containers and different from the first supply container, to the second arrangement container included in the one or more arrangement containers and different from the first arrangement container. In the above-described control device, the trajectory information generating unit may perform the process of generating the trajectory information representing the first work in accordance with the first order information acquired by the order information acquiring unit. In the above-described control device, the trajectory information generating unit may perform the process of generating the trajectory information representing the second work in accordance with the second order information acquired by the order information acquiring unit.

In a second aspect of the present invention, a picking system is provided. The above-described picking system includes the above-described control device, for example. The above-described picking system includes the above-described picking apparatus, for example.

In a third aspect of the present invention, a distribution system is provided. The above-described distribution system includes the above-described control device, for example. The above-described distribution system includes the above-described picking apparatus, for example. The above-described distribution system includes a conveying apparatus to convey one or more works, for example.

In a fourth aspect of the present invention, a tangible, non-transitory computer-readable medium having processor instructions stored thereon is provided. When executed cause the processor to perform operations for controlling a picking apparatus that moves, by using a picking hand, one or more works from one or more supply containers to one or more arrangement containers, the operations comprises:

generating trajectory information that defines a trajectory for which the picking hand picks one of the one or more works stored in one supply container included in the one or more supply containers and arranges the work in one arrangement container included in the one or more arrangement containers, wherein

the generating trajectory information includes generating, after performing a process of generating trajectory information representing a first work included in the one or more works and before a movement of the first work is completed, trajectory information representing a second work, included in the one or more works and different from the first work, by using an operation result in the process of generating the trajectory information representing the first work.

In a fifth aspect of the present invention, a control method is provided. The above-described control method may be a control method for controlling a picking apparatus. The above-described control method may be performed by a computer. In above-described control method, the picking apparatus may move the one or more works from one or more supply containers to one or more arrangement containers by using a picking hand. The above-described control method includes a trajectory information generating step of generating trajectory information that prescribes a trajectory for which the picking hand picks one of one or more works stored in one supply container included in one or more supply containers and arranges the work in one arrangement container included in one or more arrangement containers, for example. In the above-described control method, after performing the process of generating the trajectory information representing the first work included in the one or more works and before the movement of the first work is completed, the trajectory information generating step may include a step of generating the trajectory information representing the second work, included in the one or more works and different from the first work, by using an operation result in the process of generating the trajectory information representing the first work.

In a sixth aspect of the present invention, a method of producing a container is provided. The above-described method may also be a method of producing one or more arrangement container in which one or more works are arranged by using a picking apparatus. In the above-described method, the picking apparatus may move the one or more works from the one or more supply containers to one or more arrangement containers by using the picking hand. The above-described method may include a trajectory information generating step of generating trajectory information that prescribes a trajectory for which the picking hand picks one of one or more works stored in one supply container included in one or more supply containers and arranges the work in one arrangement container included in one or more arrangement containers, for example. In the above-described method, after performing the process of generating the trajectory information representing the first work included in the one or more works and before the movement of the first work is completed, the trajectory information generating step may include a step of generating the trajectory information representing a second work, included in the one or more works and different from the first work, by using an operation result in the process of generating the trajectory information representing the first work.

The summary clause does not necessarily describe all necessary features of the embodiments of the present invention. The present invention may also be a sub-combination of the features described above.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically shows one example of a system configuration of a distribution management system 100.

FIG. 2 schematically shows one example of an internal configuration of an article management apparatus 110.

FIG. 3 schematically shows one example of an internal configuration of a data registration apparatus 120.

FIG. 4 schematically shows one example of an internal configuration of a robot 140.

FIG. 5 schematically shows one example of an internal configuration of a transfer control apparatus 160.

FIG. 6 schematically shows one example of an internal configuration of a recognition unit 520.

FIG. 7 schematically shows one example of an internal configuration of a planning unit 530.

FIG. 8 schematically shows another example of an internal configuration of an execution unit 540.

FIG. 9 schematically shows one example of information processing in a transfer control apparatus 160.

FIG. 10 schematically shows one example of the recognition processing in S912.

FIG. 11 schematically shows one example of planning processing n S914.

FIG. 12 schematically shows one example of trajectory planning processing in S1160.

FIG. 13 schematically shows one example of interference checking processing in S1208.

FIG. 14 schematically shows one example of a data table 1400.

FIG. 15 schematically shows one example of concurrent processing or parallel processing between works.

FIG. 16 schematically shows one example of concurrent processing or parallel processing between works.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, (some) embodiment(s) of the present invention will be described. The embodiment(s) do(es) not limit the invention according to the claims, and all the combinations of the features described in the embodiment(s) are not necessarily essential to means provided by aspects of the invention. Note that in the drawings, the same or similar portions are provided with the same reference signs and redundant descriptions may be omitted.

[Summary of Distribution Management System 100]

FIG. 1 schematically shows one example of a system configuration of a distribution management system 100. In the present embodiment, the distribution management system 100 includes an article management apparatus 110, a data registration apparatus 120, a transfer apparatus 130 and a conveying apparatus 170. In the present embodiment, the transfer apparatus 130 includes a robot 140, an image capturing apparatus 151, an image capturing apparatus 152, an image capturing apparatus 154, an image capturing apparatus 155 and a transfer control apparatus 160. In the present embodiment, the conveying apparatus 170 includes a conveyor 171 that conveys a supply pallet 11, a conveyor 172 that conveys a supply pallet 12, a conveyor 174 that conveys a delivery pallet 14, and a conveyor 175 that conveys a delivery pallet 15.

The supply pallet 11 and the supply pallet 12 may be one example of the first container and the supply container. The delivery pallet 14 and the delivery pallet 15 may be one example of the second container and the arrangement container. The distribution management system 100 may be one example of a control device, an information processing apparatus, a picking system and a distribution system. The article management apparatus 110 may be one example of a storage apparatus. The data registration apparatus 120 may be one example of the information processing apparatus. The transfer apparatus 130 may be one example of the picking apparatus and the picking system. The robot 140 may be one example of the picking apparatus. The image capturing apparatus 151 and the image capturing apparatus 152 may be one example of an image acquiring unit. The image capturing apparatus 154 and the image capturing apparatus 155 may be one example of the image acquiring unit. The transfer control apparatus 160 may be one example of the control device and the picking control device.

In the present embodiment, the distribution management system 100 manages warehousing, storage and shipping of a plurality of types of articles. For example, the distribution management system 100 produces a piece of baggage for delivery by putting a stored article in a container for delivery based on an order from a user.

The baggage for delivery is produced according to the following procedures, for example. First, the conveyor 171 of the conveying apparatus 170 conveys the supply pallet 11, in which one article ordered by a user is stored, to a working position of the transfer apparatus 130. Similarly, the conveyor 172 of the conveying apparatus 170 conveys the supply pallet 12, in which another articles ordered by the user is stored, to the working position of the transfer apparatus 130.

In one embodiment, at least one of the supply pallet 11 and the supply pallet 12 stores one article. In another embodiment, at least one of the supply pallet 11 and the supply pallet 12 stores a plurality of articles which are of the same type. In still another embodiment, at least one of the supply pallet 11 and the supply pallet 12 stores a plurality of types of articles.

Also, the conveyor 174 of the conveying apparatus 170 conveys the delivery pallet 14 to the working position of the transfer apparatus 130. Similarly, the conveyor 175 of the conveying apparatus 170 conveys the delivery pallet 15 to the working position of the transfer apparatus 130. In the present embodiment, the delivery pallet 14 stores an article ordered by an order A (the order may be referred to as a user). The delivery pallet 15 stores an article ordered by an ordered B.

Next, the transfer apparatus 130 identifies a work 10 that is to be a working target of the transfer apparatus 130 from among the articles stored in the supply pallet 11 or the supply pallet 12. Also, the transfer apparatus 130 sets a specific position or any position of the delivery pallet 14 or the delivery pallet 15 as an arrangement position of the above-described work 10. The transfer apparatus 130 determines a trajectory for moving the identified work 10 to the specific position or any position of the delivery pallet 14 or the delivery pallet 15. The transfer apparatus 130 (i) picks the identified work 10 and takes out the identified work 10 from the supply pallet 11 or the supply pallet 12, and (ii) arranges the identified work 10 on a specific position or any position of the delivery pallet 14 or the delivery pallet 15.

If the other article ordered by the user A or the user B is stored in a current supply pallet 11 or supply pallet 12, the transfer apparatus 130 repeats a similar working to the above. If the other article ordered by the user A or the user B is stored in another supply pallet, the conveying apparatus 170 moves at least one of the current supply pallet 11 and supply pallet 12 out of the working position of the transfer apparatus 130 and conveys the above-described other supply pallet that stores the above-described other article to the working position of the transfer apparatus 130. The transfer apparatus 130 repeats the similar working to the above with respect to the other article stored in the other supply pallet.

Then, as all articles ordered by the user A are stored in the delivery pallet 14, the conveyor 174 moves the delivery pallet 14 out of the working position of the transfer apparatus 130. Similarly, as all articles ordered by the user B are stored in the delivery pallet 15, the conveyor 175 moves the delivery pallet 15 out of the working position of the transfer apparatus 130. Accordingly, the baggage for delivery can be prepared.

[Summary of Each Unit of Distribution Management System 100]

In the present embodiment, the article management apparatus 110 manages warehousing, storage and shipping of articles. The article management apparatus 110 acquires user order information that indicates order contents of a user. Details of the article management apparatus 110 are described below.

In the present embodiment, the data registration apparatus 120 acquires various information representing warehoused articles. For example, the data registration apparatus 120 acquires information representing appearances, shapes, specifications, handling methods and the like. The data registration apparatus 120 may register the acquired information in the article management apparatus 110. The data registration apparatus 120 may also register information about the all warehoused articles, and may also register information per type of articles. Details of the data registration apparatus 120 are described below.

In the present embodiment, the transfer apparatus 130 moves the work 10 from the supply pallet arranged on the working position of the transfer apparatus 130 to the delivery pallet arranged on the working position of the transfer apparatus 130. In the present embodiment, on the working position of the transfer apparatus 130, one or more supply pallets and one or more delivery pallets are arranged. The transfer apparatus 130 moves a plurality of works between the one or more supply pallets and the one or more delivery pallets.

To simplify the description, in the present embodiment, a case in which a maximum of two supply pallets and two delivery pallets are arranged on the working positions of the transfer apparatus 130 is described. However, the transfer apparatus 130 is not limited to the present embodiment. In another embodiment, on the working position of the transfer apparatus 130, three or more supply pallets may also be arranged, and three delivery pallets may also be arranged. Also, a specific configuration of the conveying apparatus 170 is not particularly limited. For example, structures and arrangements of a conveyor and an extrusion apparatus that transfers a pallet from one conveyor to another conveyor are properly designed according to purposes or applications.

In the present embodiment, the robot 140 moves a work stored in the supply pallet to a delivery pallet based on an instruction from the transfer control apparatus 160. For example, the robot 140 moves, by using the picking hand, one or more works from one or more supply pallets to one or more delivery pallets. The robot 140 may also arrange, on a second position, the work 10 picked on a first position by using the picking hand, the second position being different from the first position. The first position may be a specific position or any position of the supply pallet 11 or supply pallet 12. The second position may be a specific position or any position of the delivery pallet 14 or delivery pallet 15. Details of the robot 140 are described below. Note that the expression “one or more” may be referred to as the expression “one or more”.

In the present embodiment, if the robot 140 is to transfer a plurality of works sequentially, the robot 140 may transfer the plurality of works in any sequence. In one embodiment, for example, the robot 140 moves one work from the supply pallet 11 to the delivery pallet 14 and then moves another work from the supply pallet 11 to the delivery pallet 15. In another embodiment, for example, the robot 140 moves one work from the supply pallet 11 to the delivery pallet 14 and then moves another work from the supply pallet 12 to the delivery pallet 14. In still another embodiment, the robot 140 moves one work from the supply pallet 11 to the delivery pallet 14 and then moves another work from the supply pallet 12 to the delivery pallet 15, for example.

In the present embodiment, the image capturing apparatus 151 and the image capturing apparatus 152 respectively capture images of the supply pallet 11 and the supply pallet 12 arranged on the working position and the transfer apparatus 130. At least one of the image capturing apparatus 151 and the image capturing apparatus 152 may also acquire three-dimensional information (for example, a three-dimensional point cloud) of an image of a surface of a supply pallet that is to be an image capturing target. At least one of the image capturing apparatus 151 and the image capturing apparatus 152 may also capture a stereo image of the supply pallet that is to be the image capturing target. At least one of the image capturing apparatus 151 and the image capturing apparatus 152 may also acquire temperature information of the supply pallet that is to be the image capturing target. As the image capturing apparatus 151 and the image capturing apparatus 152, a camera, a stereo camera, a three-dimensional sensor and the like can be illustrated. At least one of the image capturing apparatus 151 and the image capturing apparatus 152 may also include a plurality of image capturing apparatuses.

In the present embodiment, the image capturing apparatus 154 and the image capturing apparatus 155 respectively capture images of the delivery pallet 14 and the delivery pallet 15 arranged on the working position of the transfer apparatus 130. At least one of the image capturing apparatus 154 and the image capturing apparatus 155 may also acquire three-dimensional information (for example, a three-dimensional point cloud) of a surface of a delivery pallet that is to be an image capturing target. At least one of the image capturing apparatus 154 and the image capturing apparatus 155 may also capture a stereo image of the delivery pallet being the image capturing target. At least one of the image capturing apparatus 154 and the image capturing apparatus 155 may also acquire the temperature information of the delivery pallet being the image capturing target. As the image capturing apparatus 154 and the image capturing apparatus 155, a camera, a stereo camera, a three-dimensional sensor and the like can be illustrated. At least one of the image capturing apparatus 154 and the image capturing apparatus 155 may also include a plurality of image capturing apparatuses.

In the present embodiment, the transfer control apparatus 160 controls the transfer apparatus 130. In one embodiment, the transfer control apparatus 160 is arranged outside the transfer apparatus 130. In another embodiment, the transfer control apparatus 160 is arranged inside the transfer apparatus 130. Details of the transfer control apparatus 160 are described below.

In the present embodiment, the conveying apparatus 170 conveys one or more works. For example, the conveying apparatus 170 conveys one or more works stored in the supply pallet 11 or the supply pallet 12. The conveying apparatus 170 may also convey one or more works stored in the delivery pallet 14 or the delivery pallet 15.

[Specific Configuration of Each Unit of Distribution Management System 100]

Each unit of the distribution management system 100 may also be realized by hardware, may also be realized by software, and may also be realized by hardware and software. At least some of the respective units of the distribution management system 100 may also be realized by a single server, or may also be realized by a plurality of servers. At least some of the respective units of the distribution management system 100 may also be realized on a virtual server or a cloud system. At least some of the respective units of the distribution management system 100 may also be realized by a personal computer or a mobile terminal. As the mobile terminal, a mobile phone, a smartphone, a PDA, a tablet, a notebook computer or a laptop computer, a wearable computer and the like can be illustrated. Each unit of the distribution management system 100 may also store information by using a distributed inventory technology such as a block chain, or a distributed network.

In a case in which at least some of the components configuring the distribution management system 100 are realized by software, the component realized by the software may be realized by activating software or a program prescribing an operation regarding the component in an information processing apparatus in a general configuration. The above-described information processing apparatus in the general configuration may include: (i) a data processing apparatus having a processor, such as CPU and GPU, a ROM, a RAM, a communication interface and the like; (ii) an input apparatus such as a keyboard, a pointing device, a touch panel, a camera, a sound input apparatus, a gesture input apparatus, various sensors and a GPS receiver; (iii) an output apparatus such as a display apparatus, a sound output apparatus and a vibration apparatus; and (iv) a storage apparatus (including an external storage apparatus) such as a memory, HDD and SSD.

In the above-described information processing apparatus in the general configuration, the above-described data processing apparatus or storage apparatus may store the above-described software or program. The above-described software or program is performed by the processor, and accordingly, causes the above-described information processing apparatus to perform an operation prescribed by the software or program. The above-described software or program may also be stored in a non-transitory computer-readable medium. The above-described software or program may be a program to cause a computer to serve as the transfer control apparatus 160 or a part of the transfer control apparatus 160.

The above-described software or program may be a program to control the picking apparatus. In the above-described program, the picking apparatus may arrange, on a second position, a work picked on a first position by using a picking hand, the second position being different from the above-described first position. The above-described program may be a program to cause a computer to perform a first candidate information acquiring procedure of acquiring first candidate information indicating each of one or more postures that the above-described picking hand can take on the above-described first position. The above-described program may be a program to cause a computer to perform a second candidate information acquiring procedure of acquiring second candidate information indicating each of one or more postures that the above-described picking hand can take on the above-described second position, and an extracting procedure of extracting one combination from among one or more combinations regarding the one or more postures indicated by the above-described first candidate information and the one or more postures indicated by the above-described second candidate information. The above-described program may be a program to cause a computer to perform a trajectory calculating procedure of calculating a trajectory for which (i) the above-described picking hand picks, on the above-described first position, the above-described work with a posture associated with the above-described first position included in the above-described one combination, and (ii) the above-described picking hand arranges, on the above-described second position, the above-described work with a posture associated with the above-described second position included in the above-described one combination. The above-described program may be a program to cause a computer to perform a determining procedure of determining whether an interference is present or absent on the trajectory that is calculated in the above-described trajectory calculating procedure.

The above-described software or program may be a program to control the picking apparatus. In the above-described program, the picking apparatus may arrange, on a second position of a second container, a work picked on a first position of a first container by using a picking hand, the second container being different from the above-described first container. The above-described program may be a program to cause a computer to perform a trajectory information generating procedure of generating trajectory information defining a trajectory for which (i) the above-described picking hand picks, on the above-described first position, the above-described work with a posture associated with the above-described first position included in the above-described one combination, and (ii) the picking hand arranges, on the above-described second position, the above-described work with a posture associated with the above-described second position included in the above-described one combination. The above-described program may be a program to cause a computer to perform an execution control procedure to operate the above-described picking apparatus based on the trajectory information generated in the above-described trajectory information procedure. The above-described program may be a program to cause a computer to perform an execution time estimating procedure of estimating a period of time that is from a time when the above-described picking apparatus receives an instruction for starting an operation on the above-described work to a time when the operation of the above-described picking apparatus on the above-described work is ended in the above-described execution control procedure. In the above-described program, the above-described trajectory information generating procedure may also include a procedure of adjusting the amount of calculation based on an estimation result obtained in the above-described execution time estimating procedure.

The above-described software or program may be a program to control the picking apparatus. In the above-described program, the picking apparatus may arrange, on a second position of a second container, a work picked on a first position of a first container by using a picking hand, the second container being different from the above-described first container. The above-described program may be a program to cause a computer to perform a point cloud information acquiring procedure of acquiring point cloud information of a region including at least one of the above-described picking hand and the above-described work at a timing being at least one of (i) a timing when the above-described picking hand picks the above-described work on the above-described first position, (ii) a timing when the above-described picking hand moves the above-described work out of the above-described first container, (iii) a timing when the above-described picking hand moves the above-described work in the above-described second container, and (iv) a timing when the above-described picking hand arranges the above-described work on the above-described second position. The above-described program may be a program to cause a computer to perform an abnormality detection procedure of detecting abnormality based on the point cloud information acquired in the above-described point cloud information acquiring procedure.

The above-described software or program may be a program to control the picking apparatus. In the above-described program, the picking apparatus may move, by using a picking hand, one or more works from one or more supply containers to one or more arrangement containers. The above-described program may be a program to cause a computer to perform a trajectory information generating procedure of generating trajectory information defining a trajectory for which the above-described picking hand picks one of the above-described one or more works stored in one supply container included in the above-described one or more supply containers and arranges the work of the above-described one or more works to one arrangement container included in the above-described one or more arrangement containers. In the above-described program, after performing a process of generating trajectory information representing a first work included in the above-described one or more works and before completing moving the above-described first work, the above-described trajectory information generating procedure may include a procedure of generating trajectory information representing a second work, included in the above-described one or more works and different from the above-described first work, by using an operation result in the process of generating the trajectory information representing the above-described first work.

The above-described software or program may be a program to cause a computer to perform a three-dimensional information acquiring procedure of acquiring three-dimensional information of a target object. The above-described software or program may be a program to cause a computer to perform a holding information acquiring procedure of acquiring holding information that defines a method of holding the above-described target object by using the picking hand. The above-described software or program may be a program to cause a computer to perform an information management procedure of associating (i) three-dimensional information of the above-described target object acquired by the above-described three-dimensional information acquiring unit with (ii) holding information acquired by the above-described holding information acquiring unit, and storing the information in a storage apparatus.

FIG. 2 schematically shows one example of an internal configuration of an article management apparatus 110. In the present embodiment, the article management apparatus 110 includes an order information acquiring unit 210, a conveying apparatus management unit 220, a transfer apparatus management unit 230, a request processing unit 240 and an article information storing unit 250. The order information acquiring unit 210 may be one example of an order information acquiring unit. The article information storing unit 250 may be one example of the storage apparatus.

In the present embodiment, the order information acquiring unit 210 acquires user order information indicating order contents of a user. The user order information may include identification information of an article that is to be a working target of the transfer apparatus 130. As the identification information of the article, a name of the article, a code number added to the article, a feature of an appearance of the article and the like can be illustrated. The user order information may be one example of the order information.

In one embodiment, the order information acquiring unit 210 receives an order from a user via an input apparatus (not shown in the drawings) such as a keyboard, a touch panel, a sound input apparatus and a gesture input apparatus that are arranged in the article management apparatus 110. In another embodiment, the order information acquiring unit 210 receives the user order information via a communication network from a communication terminal or another information processing apparatus of a user. The communication network may be a wired communication transmission path, may be a wireless communication transmission path, and may also be a combination of a wireless communication transmission path and a wired communication transmission path.

In the present embodiment, the conveying apparatus management unit 220 manages the conveying apparatus 170. In one embodiment, the conveying apparatus management unit 220 transfers the user order information acquired by the order information acquiring unit 210 to the conveying apparatus 170. In another embodiment, the conveying apparatus management unit 220 may analyze the user order information from one or more users to check inventory states of articles included in the user order information. In still another embodiment, the conveying apparatus management unit 220 may analyze the user order information from one or more users, and if multiple types of articles are included in the user order information, the conveying apparatus management unit 220 may determine a conveying sequence of the multiple types of articles to the transfer apparatus 130 based on an inventory position of each article such that the entire conveying efficiency satisfies a predetermined condition.

In the present embodiment, the transfer apparatus management unit 230 manages the transfer apparatus 130. In one embodiment, the transfer apparatus management unit 230 may analyze the user order information, and if multiple types of articles are included in the user order information, the transfer apparatus management unit 230 may determine a picking sequence of the multiple types of articles to the transfer apparatus 130 such that working efficiency of the transfer apparatus 130 satisfies a predetermined condition. If the conveying apparatus management unit 220 determines the conveying sequence of the multiple types of articles to the transfer apparatus 130, the transfer apparatus management unit 230 may determine the picking sequence of the multiple types of articles to the transfer apparatus 130 while taking the conveying sequence into account. In another embodiment, the transfer apparatus management unit 230 transfers the user order information acquired by the order information acquiring unit 210 to the transfer apparatus 130. In this case, the transfer apparatus 130 may also analyze the user order information to determine the picking sequence of the multiple types of articles to the transfer apparatus 130.

In the present embodiment, the request processing unit 240 receives various requests. The request processing unit 240 processes the received requests. In one embodiment, the request processing unit 240 receives a registration request of data regarding an article from the data registration apparatus 120. As the request processing unit 240 receives the above-described registration request, the request processing unit 240 checks, for example, whether or not the data of the article to be registered has already been stored in the article information storing unit 250. If the above-described data is not stored in the article information storing unit 250, the request processing unit 240 stores the above-described data in the article information storing unit 250. In a case where the above-described data has been stored in the article information storing unit 250, if the contents of the data have been changed the request processing unit 240 updates the above-described data. On the other hand, the request processing unit 240 ends the processing if the contents of the data are not changed.

In another embodiment, the request processing unit 240 receives, from the transfer control apparatus 160, a sending request of data matched with a specific condition. As the request processing unit 240 receives the above-described sending request, the request processing unit 240 accesses, for example, to the article information storing unit 250 to check whether the data matched with the condition is present or absent. If the data matched with the above-described condition has been stored in the article information storing unit 250, the request processing unit 240 sends the data to the transfer control apparatus 160. If the data matched with the above-described condition is not stored in the article information storing unit 250, the request processing unit 240 sends information indicating that no data exists to the transfer control apparatus 160.

In the present embodiment, the article information storing unit 250 stores various information representing an article. The article information storing unit 250 may store the information acquired by the data registration apparatus 120.

FIG. 3 schematically shows one example of an internal configuration of the data registration apparatus 120. In the present embodiment, the data registration apparatus 120 includes a three-dimensional information acquiring unit 312, a two-dimensional information acquiring unit 314, a mass information acquiring unit 316 and an article information registration unit 320. In the present embodiment, the article information registration unit 320 includes an input/output unit 330, a registration information generating unit 340 and a registration request sending unit 350. In the present embodiment, the registration information generating unit 340 includes a coordinate information adding unit 342, a holding information adding unit 344 and a specification information adding unit 346.

The article information registration unit 320 may be one example of the information management unit. The holding information adding unit 344 may be one example of the holding information acquiring unit. The specification information adding unit 346 may be one example of the feature information acquiring unit. The registration request sending unit 350 may be one example of the information management unit.

In the present embodiment, the three-dimensional information acquiring unit 312 acquires three-dimensional information of a target article that is to be a target of data registration processing. The three-dimensional information may include at least one of home position information and coordinate axis information. The three-dimensional information may also include CAD data of the target article. The three-dimensional information may also include data obtained by three-dimensionally measuring the target article. The target article may be one example of the target object. The three-dimensional information acquiring unit 312 may also be (i) a three-dimensional measurement apparatus in any manner and may also be (ii) a communication interface or a storage apparatus to receive three-dimensional information from the three-dimensional measurement apparatus or another information processing apparatus.

In the present embodiment, the two-dimensional information acquiring unit 314 acquires two-dimensional information of a target article that is to be a target of the data registration processing. The two-dimensional information may include CAD data of the target article. The two-dimensional information may include image data of three or more images obtained by capturing the images of the target article from three or more directions. The two-dimensional information may include image data of four or more images obtained by capturing the images of the target article from four or more directions. The two-dimensional information may include image data of five or more images obtained by capturing the images of the target article from five or more directions. The two-dimensional information may include image data of six or more images obtained by capturing the images of the target article from six or more directions. The two-dimensional information acquiring unit 314 may also be (i) an image capturing apparatus in any manner, and may also be (ii) a communication interface or a storage apparatus to receive two-dimensional information from the image capturing apparatus or another information processing apparatus.

In the present embodiment, the mass information acquiring unit 316 acquires mass information indicating the mass of the target article. The mass information may also include information of a center of gravity indicating a position of the center of gravity of the target article. The mass information acquiring unit 316 may also be (i) a mass measurement apparatus in any manner, and may also be (ii) a communication interface or a storage apparatus to receive mass information from the mass measurement apparatus or another information processing apparatus.

In the present embodiment, the article information registration unit 320 stores, in the article information storing unit 250, (i) the three-dimensional information of the target article acquired by the three-dimensional information acquiring unit 312 and (ii) various information representing the target article in association with each other. For example, the article information registration unit 320 sends, to the article management apparatus 110, a request for storing (i) the three-dimensional information of the target article acquired by the three-dimensional information acquiring unit 312 and (ii) various information representing the target article in the article information storing unit 250.

In the present embodiment, the input/output unit 330 receives an input from a user. Also, the input/output unit 330 outputs information to the user. The input/output unit 330 includes an input apparatus such as a keyboard, a pointing device, a touch panel, a camera, a sound input apparatus, a gesture input apparatus, various sensors and a GPS receiver, for example. The input/output unit 330 may also be a communication interface with an external storage apparatus. The input/output unit 330 may also be a communication interface for sending and receiving information to/from another information processing apparatus via a communication line. Accordingly, information representing a target article can be acquired by accessing to a website in which a manufacturer or seller of the target article provides a service, for example. The input/output unit 330 includes an output apparatus such as a display apparatus, a sound output apparatus and a vibration apparatus, for example. Accordingly, an input screen for the user to input the information representing the target article can be presented to the user, for example.

In the present embodiment, the registration information generating unit 340 generates information that is to be registered in the article management apparatus 110. For example, the registration information generating unit 340 adds various information to at least one of the three-dimensional information of the target article acquired by the three-dimensional information acquiring unit 312, the two-dimensional information of the target article acquired by the two-dimensional information acquiring unit 314 and the mass information of the target article acquired by the mass information acquiring unit 316, to generate the information to be registered in the article management apparatus 110.

In the present embodiment, the coordinate information adding unit 342 adds at least one of the home position information and the coordinate axis information to at least one of the three-dimensional information of the target article acquired by the three-dimensional information acquiring unit 312 and the two-dimensional information of the target article acquired by the two-dimensional information acquiring unit 314. The coordinate information adding unit 342 may also acquire at least one of the home position information and the coordinate axis information from the input/output unit 330.

In the present embodiment, the holding information adding unit 344 adds holding information defining a method of holding the target article by using the picking hand to at least one of the three-dimensional information of the target article acquired by the three-dimensional information acquiring unit 312 and the two-dimensional information of the target article acquired by the two-dimensional information acquiring unit 314.

In one embodiment, the holding information adding unit 344 determines a method of holding the target article by acquiring the holding information from the input/output unit 330. In another embodiment, the holding information adding unit 344 determines the method of holding the target article based on (i) a shape of the target article and (ii) information associating a typical shape of the article with the method of holding the article.

In the present embodiment, the specification information adding unit 346 adds specification information indicating a specification of the target article to at least one of the three-dimensional information of the target article acquired by the three-dimensional information acquiring unit 312 and the two-dimensional information of the target article acquired by the two-dimensional information acquiring unit 314. As the specification of the target article, (i) a name of the article, (ii) an identification code added to the article, (iii) a dimension of the article, (iv) the mass of the article, (v) matters regarding storage, transportation or usage of the article and the like can be illustrated.

In one embodiment, the specification information adding unit 346 may acquire the specification information from the input/output unit 330. In another embodiment, the specification information adding unit 346 may acquire, based on at least one of the three-dimensional information acquired by the three-dimensional information acquiring unit 312 and the two-dimensional information acquired by the two-dimensional information acquiring unit 314, feature information for recognizing the target article and include the feature information in the specification information. The specification information adding unit 346 may use a method used in an image recognition field such as face recognition and living body recognition to acquire the feature information.

The feature information may be information representing a feature of at least parts of the appearance of the target article. As the feature of the appearance of the target article, (i) a feature regarding at least one of a shape, a pattern and a color of the target article, (ii) a logo mark added to a target article, a name of the article or an identification code to identify the article and the like can be illustrated.

Accordingly, the registration information generating unit 340 can generate the information that is to be registered in the article management apparatus 110. Note that the information that is to be registered in the article management apparatus 110 may at least include at least one of the three-dimensional information of the target article acquired by the three-dimensional information acquiring unit 312 and the two-dimensional information of the target article acquired by the two-dimensional information acquiring unit 314, or may not include the mass information, the coordinate information, the holding information and the specification information.

In one embodiment, the information to be registered in the article management apparatus 110 includes the three-dimensional information and the holding information of the target article. The information to be registered in the article management apparatus 110 may also include (i) the three-dimensional information and the holding information of the target article and (ii) at least one of the feature information and the identification information of the target article. The information to be registered in the article management apparatus 110 may also include (i) the three-dimensional information and the holding information of the target article and (ii) at least one of the mass information, the coordinate information and the specification information of the target article. The information to be registered in the article management apparatus 110 may also include (i) the three-dimensional information and the holding information of the target article, (ii) at least one of the feature information and the identification information of the target article, and (iii) at least one of the mass information, the coordinate information and the specification information of the target article.

In another embodiment, the information to be registered in the article management apparatus 110 includes the three-dimensional information, the two-dimensional information and the holding information of the target article. The information to be registered in the article management apparatus 110 may also include (i) the three-dimensional information, the two-dimensional information and the holding information of the target article, and (ii) at least one of the feature information and the identification information of the target article. The information that is to be registered in the article management apparatus 110 may also include (i) the three-dimensional information, the two-dimensional information and the holding information of the target article, and (ii) at least one of the mass information, the coordinate information and the specification information of the target article. The information that is to be registered in the article management apparatus 110 may also include (i) the three-dimensional information, the two-dimensional information and the holding information of the target article, (ii) at least one of the feature information and the identification information of the target article, and (iii) at least one of the mass information, the coordinate information and the specification information of the target article.

In the present embodiment, the registration request sending unit 350 stores the registration information generated by the registration information generating unit 340 in the article management apparatus 110. For example, the registration request sending unit 350 sends, to the article management apparatus 110, a registration request for storing the registration information generated by the registration information generating unit 340 in the article information storing unit 250.

In the present embodiment, a case in which the article information storing unit 250 that is where the registration information is to be registered is arranged in the article management apparatus 110 is described. However, the article information storing unit 250 is not limited to the present embodiment. In another embodiment, the article information storing unit 250 may also be arranged in the data registration apparatus 120.

FIG. 4 schematically shows one example of an internal configuration of a robot 140. In the present embodiment, the robot 140 includes a manipulator 410, a driving unit 420 and a drive control unit 430. In the present embodiment, the manipulator 410 includes a picking hand 412 and a robot arm 414.

In the present embodiment, the picking hand 412 holds the work 10. The holding manner of the picking hand 412 is not particularly limited. As the holding manner of the picking hand 412, a manner of attracting the work 10 with a low pressure, a manner of holding the work 10 by a plurality of finger modules and the like can be illustrated. The picking hand 412 may also be exchanged in accordance with the working contents. The picking hand 412 may have various sensors such as a contact sensor, a mass measurement sensor and a load measurement sensor.

In the present embodiment, the robot arm 414 is coupled with the picking hand 412 to adjust a position and a posture of the picking hand 412. Although the structure of the robot arm 414 is not particularly limited, a multijoint robot arm having a plurality of joints is used, for example. The robot arm 414 may have various sensors such as a mass measurement sensor and a load measurement sensor. The load measurement sensor may be a sensor to measure torque, a current value or a voltage value of the driving unit 420 that drives each joint of the robot arm 414.

In the present embodiment, the driving unit 420 drives the manipulator 410. In one embodiment, the driving unit 420 drives the picking hand 412. For example, the driving unit 420 supplies air to the picking hand 412, or sucks air from the picking hand 412. In another embodiment, the driving unit 420 drives the robot arm 414. For example, the driving unit 420 adjusts an angle of each joint of the robot arm 414. The driving unit 420 may be a motor. The driving unit 420 may have a plurality of motors.

In the present embodiment, the drive control unit 430 controls the driving unit 420. For example, the drive control unit 430 receives, from the transfer control apparatus 160, an instruction for controlling an operation of the robot 140. The drive control unit 430 controls the driving unit 420 based on the instruction from the transfer control apparatus 160. The instruction for controlling the operation of the robot 140 may be information indicating a trajectory of the picking hand 412.

As the information indicating the trajectory of the picking hand 412, (i) information indicating an initial arrangement, a passing arrangement, a final arrangement and move time of the picking hand 412, (ii) information indicating an angle of each joint and move time of the robot arm 414 in the initial arrangement, the passing arrangement and the final arrangement, (iii) information indicating an arrangement of the picking hand 412 at each time during the move time, (iv) information indicating an angle of each joint of the robot arm 414 at each time during the move time, and the like can be illustrated. The arrangement of the picking hand 412 may be specified by the position and the posture of the picking hand 412, for example.

FIG. 5 schematically shows one example of an internal configuration of the transfer control apparatus 160. In the present embodiment, the transfer control apparatus 160 includes a recognition information storing unit 510, a recognition unit 520, a planning unit 530, an execution unit 540 and a concurrent processing control unit 550.

The recognition information storing unit 510 may be one example of the holding information storing unit. The recognition unit 520 may be one example of the order information acquiring unit, the image acquiring unit, the point cloud information acquiring unit, the holding information extracting unit and the container information acquiring unit. The planning unit 530 may be one example of the control device and the trajectory information generating unit. The execution unit 540 may be one example of the control device and the execution control unit. The concurrent processing control unit 550 may be one example of the execution time estimating unit and the constraint condition setting unit.

In the present embodiment, the recognition information storing unit 510 stores various information representing the recognition processing in the recognition unit 520. In one embodiment, the recognition information storing unit 510 stores information that is used for the recognition processing in the recognition unit 520. For example, the recognition information storing unit 510 stores the identification information to identify each of one or more types of articles and at least one of the three-dimensional information, the two-dimensional information and the feature information of each article in association with each other. Accordingly, the recognition unit 520 can recognize articles stored in the supply pallet and recognize a work 10 that is to be a working target of the transfer apparatus 130 from among the identified articles.

The recognition information storing unit 510 may further store at least one of the home position information, the coordinate axis information, the holding information and the specification information of each article. These pieces of information are used by the planning unit 530, for example. For example, the recognition information storing unit 510 stores the identification information to identify each of one or more types of articles, at least one of the three-dimensional information, the two-dimensional information and the feature information of each article, and at least one of the home position information, the coordinate axis information, the holding information and the specification information of each article in association with each other.

In another embodiment, the recognition information storing unit 510 stores the information obtained by the recognition processing in the recognition unit 520. For example, the recognition information storing unit 510 stores (i) a recognition result of each of one or more supply pallets arranged on working positions of the transfer apparatus 130, (ii) a recognition result of an article stored in each of the one or more supply pallets, (iii) a recognition result of a work 10 that is to be the working target of the transfer apparatus 130 among the recognized articles, (iv) a point cloud of surfaces of the one or more supply pallets, (v) a recognition result of each of one or more delivery pallets arranged on working positions of the transfer apparatus 130, (vi) a recognition result of an article stored in each of the one or more delivery pallets, (vii) a recognition result of the being-transferred work 10 on each of the one or more delivery pallets, (viii) a point cloud of the surfaces of the one or more delivery pallets, and the like. The above-described point cloud may be a three-dimensional point cloud.

The recognition information storing unit 510 may store information required for the working of the transfer apparatus 130 among the information representing the articles being stored in the article information storing unit 250. For example, the recognition information storing unit 510 stores information representing the article that was previously the working target of the transfer apparatus 130, information representing the article that is often the working target of the transfer apparatus 130, information representing an article that is scheduled to be the working target of the transfer apparatus 130, information representing an article that is estimated to be the working target of the transfer apparatus 130 in a near future according to a previous working history, and the like. Accordingly, the recognition unit 520 can extract target information at high speed.

The recognition information storing unit 510 may be realized by a medium being different from the article information storing unit 250 that is where the data acquired by the data registration apparatus 120 is to be registered. For example, the recognition information storing unit 510 is realized by a medium to which the recognition unit 520 can access at higher speed than the article information storing unit 250. Accordingly, the recognition unit 520 can extract the target information at high speed.

In the present embodiment, the recognition unit 520 performs the recognition processing. For example, the planning unit 530 acquires the working information indicating the working contents of the transfer apparatus 130 from the article management apparatus 110. The working information includes the information indicating the picking sequence of the one or more articles to the transfer apparatus 130, for example. The working information may also be the user order information. The working information may be one example of the order information. Also, the recognition unit 520 acquires, from the image capturing apparatus 151, the image capturing apparatus 152. the image capturing apparatus 154 and the image capturing apparatus 155, image data of each of the one or more supply pallets and the one or more delivery pallets arranged on the working positions of the transfer apparatus 130.

In the present embodiment, the recognition unit 520 recognize, with reference to the information stored in the recognition information storing unit 510 or the article information storing unit 250, each supply pallet, the articles stored in each supply pallet, each delivery pallet, the articles stored in each delivery pallet and the work 10 being the working target of the transfer apparatus 130. The recognition unit 520 may send the information indicating the recognition result to the planning unit 530. The recognition unit 520 may send the information indicating the recognition result to the execution unit 540. The recognition unit 520 may store the information indicating the recognition result in the recognition information storing unit 510. Details of the recognition unit 520 are described below.

In the present embodiment, the planning unit 530 performs the planning processing. For example, the planning unit 530 creates a working plan of the transfer apparatus 130. More specifically, the planning unit 530 generates trajectory information defining a trajectory for which the picking hand 412 picks the work 10 stored in a specific supply pallet and arranges the work on a specific delivery pallet. The specific supply pallet may be one example of one supply container included in one or more supply containers. The specific delivery pallet may be one example of one arrangement container included in one or more arrangement containers.

The trajectory information prescribes a trajectory for which the picking hand 412 picks the work 10 on a first position and arranges the work on a second position, for example. The second position may be a position being different from the first position. The first position may be any position or a specific position of the supply pallet. The second position may be any position or a specific position of the delivery pallet.

In the present embodiment, the planning unit 530 generates the trajectory information based on the working information from the article management apparatus 110 and the recognition result of the recognition unit 520 with reference to the holding information stored in the recognition information storing unit 510 or the article information storing unit 250. Accordingly, the planning unit 530 can create a working plan of the transfer apparatus 130. The planning unit 530 may generate control information for controlling at least one of the picking hand 412 and the robot arm 414 based on the generated trajectory information. The planning unit 530 may send the generated control information to the execution unit 540. Details of the planning unit 530 are described below.

In the present embodiment, the execution unit 540 performs the execution processing. For example, the execution unit 540 receives the control information from the planning unit 530. The execution unit 540 sends the control information to the drive control unit 430 of the robot 140 to operate the robot 140 based on the trajectory information generated by the planning unit 530. Also, the execution unit 540 monitors the operation of the robot 140, the supply pallet and the delivery pallet. The control information may be one example of an instruction to the robot 140. The control information may be one example of an instruction for causing the robot 140 to start an operation on the work 10. Details of the execution unit 540 are described below.

In the present embodiment, the recognition unit 520 may perform concurrently or in parallel parts of a plurality of steps configuring the recognition processing. Similarly, the planning unit 530 may perform concurrently or in parallel parts of a plurality of steps configuring the planning processing. The execution unit 540 may perform concurrently or in parallel parts of a plurality of steps configuring the execution processing. Also, the transfer control apparatus 160 may perform concurrently or in parallel the recognition processing, the planning processing and the execution processing regarding a plurality of works 10. For example, the transfer control apparatus 160 performs planning processing of a second work and recognition processing of a third work while performing execution processing of a first work. The above-described concurrent processing or parallel processing can be realized by a pipeline processing technology, for example.

In the present embodiment, the concurrent processing control unit 550 controls the concurrent processing or parallel processing in the transfer control apparatus 160. In one embodiment, the concurrent processing control unit 550 controls the concurrent processing or parallel processing in each of the recognition unit 520, the planning unit 530 and the execution unit 540. In another embodiment, in a case in which the transfer control apparatus 160 performs concurrently or in parallel the processing regarding the plurality of works 10, the concurrent processing control unit 550 controls the recognition unit 520, the planning unit 530 and the execution unit 540 such that the information can be smoothly exchanged between the recognition unit 520, the planning unit 530 and the execution unit 540.

First Embodiment of Controlling Other Elements by Concurrent Processing Control Unit 550

In one embodiment, the concurrent processing control unit 550 estimates a period of time that is from a time when the robot 140 receives the control information for starting the operation on the work 10 to a time when the operation of the robot 140 on the work 10 is ended. For example, the trajectory information includes information representing (i) an initial position and an initial posture of the picking hand 412, (ii) a position and a posture of the picking hand 412 picking the work 10 on the supply pallet, (iii) a position and a posture of the picking hand 412 arranging the work 10 on the delivery pallet, and (iv) move time of the work 10. Here, the concurrent processing control unit 550 may estimate the above-described time based on the trajectory information generated by the planning unit 530.

The concurrent processing control unit 550 may determine a constraint condition regarding an amount or duration time of calculation of the processing in at least one of the recognition unit 520, the planning unit 530 and the execution unit 540 based on the above-described estimation result. For example, the concurrent processing control unit 550 determines the constraint condition regarding the amount or duration time of calculation of the process in at least one of a plurality of steps configuring the recognition processing in the recognition unit 520 based on the above-described estimation result. The recognition unit 520 may adjust the amount or duration time of calculation based on the above-described constraint condition. Similarly, the concurrent processing control unit 550 may determine the constraint condition regarding the amount or duration time of calculation of the process in at least one of a plurality of steps configuring the planning processing in the planning unit 530 based on the above-described estimation result. The planning unit 530 may adjust the amount or duration time of calculation based on the above-described constraint condition.

The constraint condition may be at least one of (i) the maximum value of an amount or duration time of calculation that is until progress of each processing reaches a preset milestone and (ii) the maximum value of an amount or duration time of calculation that is until each processing is completed. The above-described maximum value may be determined based on the above-described estimation result.

For example, in a case in which the transfer control apparatus 160 performs the planning processing of the second work and the recognition processing of the third work while performing the execution processing of the first work, the concurrent processing control unit 550 determines the maximum value of the amount or duration time of calculation of the planning processing of the second work such that the planning processing of the second work in the planning unit 530 is ended before the above-described estimate time is expired or elapsed. Accordingly, the trajectory information or control information representing the second work can be generated before the execution processing of the first work in the execution unit 540 is ended.

Similarly, the concurrent processing control unit 550 may determine the maximum value of the amount or duration time of calculation of the recognition processing of the third work such that the recognition processing of the third work in the recognition unit 520 is ended before the planning processing of the second work in the planning unit 530 is ended. The concurrent processing control unit 550 may also determines the maximum value of the amount or duration time of calculation of the recognition processing of the third work such that the recognition processing of the third work in the recognition unit 520 ends before the execution processing of the first work in the execution unit 540 is ended.

Second Embodiment of Controlling Other Elements by Concurrent Processing Control Unit 550

In another embodiment, the concurrent processing control unit 550 adjusts the timings between the processing in the recognition unit 520, the processing in the planning unit 530 and the processing in the execution unit 540. For example, the concurrent processing control unit 550 adjusts the timing between the recognition processing in the recognition unit 520 and the execution processing in the execution unit 540. More specifically, the concurrent processing control unit 550 adjusts the timing between a step in which the recognition unit 520 acquires the point cloud information and a step in which the execution unit 540 detects the abnormality.

For example, in a case in which the transfer control apparatus 160 performs the planning processing of the second work and the recognition processing of the third work while performing the execution processing of the first work, the execution unit 540 acquires the point cloud information of the supply pallet in which the first work is stored in periods of time that are before and after the picking hand 412 picks the first work, to determine whether the abnormality is present or absent. Also, the execution unit 540 acquires the point cloud information of the delivery pallet for which the first work is arranged in periods of time that are before and after the picking hand 412 arranges the first work, to determine whether the abnormality is present or absent. Accordingly, occurrence of the abnormality can be rapidly detected. On the other hand, in the recognition processing of the recognition unit 520, for example, the process of recognizing the third work and the process of acquiring the point cloud information of the supply pallet in which the first work is stored or the delivery pallet on which the first work is arranged are performed concurrently. Because the duration time of calculation for the process of acquiring the point cloud information is shorter compared to the process of recognizing the work, the recognition unit 520 can perform the process of acquiring the point cloud information for multiple times while performing the process of recognizing the work for one time.

Here, the concurrent processing control unit 550 controls the recognition unit 520, for example, to adjust the timing at which the recognition unit 520 starts the step of acquiring the point cloud information of the supply pallet or the delivery pallet in accordance with the progress of the execution processing of the first work by the execution unit 540. According to one embodiment, the concurrent processing control unit 550 estimates the timing at which the picking hand 412 picks the first work or estimates a timing at which the picking hand 412 arranges the first work. Then, the concurrent processing control unit 550 controls the recognition unit 520 such that the recognition unit 520 starts the step of acquiring the point cloud information in accordance with the timing. The concurrent processing control unit 550 may estimate the above-described timing based on the trajectory information generated by the planning unit 530. According to another embodiment, at the timing when the picking hand 412 picks the first work or the timing when the picking hand 412 arranges the first work, if the step of acquiring the point cloud information is being performed, the recognition unit 520 suspends the step and newly starts the process of acquiring the point cloud information.

FIG. 6 schematically shows one example of an internal configuration of the recognition unit 520. In the present embodiment, the recognition unit 520 includes a position and posture information generating unit 610 and a point cloud information generating unit 620. The position and posture information generating unit 610 may be one example of an image acquiring unit, a point cloud information acquiring unit, a container information acquiring unit and a holding information extracting unit. The point cloud information generating unit 620 may be one example of the image acquiring unit and the point cloud information acquiring unit.

In the present embodiment, the position and posture information generating unit 610 acquires, from the image capturing apparatus 151, the image capturing apparatus 152, the image capturing apparatus 154 and the image capturing apparatus 155, image data of each of the one or more supply pallets and the one or more delivery pallets arranged on the working positions of the transfer apparatus 130. The position and posture information generating unit 610 may also acquire, from the image capturing apparatus 151, the image capturing apparatus 152, the image capturing apparatus 154, and the image capturing apparatus 155, point cloud data of each of the one or more supply pallets and the one or more delivery pallets arranged on the working positions of the transfer apparatus 130. The above-described point cloud data may be three-dimensional point cloud data.

In the present embodiment, the position and posture information generating unit 610 acquires working information from the article management apparatus 110. As described above, the working information includes information indicating the picking sequence of the one or more articles by the transfer apparatus 130, for example. The working information may include identification information of an article that is to be the working target of the transfer apparatus 130. The working information, information indicating an article that is the working target, information indicating a supply pallet 12 that is the move source and information indicating a delivery pallet 14 that is the move destination may also be stored in association with each other. In a case in which the supply pallet 12 stores a single type of article, the identification information of the supply pallet 12 may also be used as the identification information of the article that is the working target of the transfer apparatus 130.

In the present embodiment, the position and posture information generating unit 610 recognizes at least one (which may be referred to as an element) of (i) the supply pallet, (ii) at least some of the one or more articles stored in the supply pallet, (iii) the delivery pallet and (iv) at least some of the one or more articles stored in the delivery pallet. The position and posture information generating unit 610 may also recognize the work 10 that is the working target of the transfer apparatus 130. The position and posture information generating unit 610 recognizes a position and a posture of each recognized element. The position and posture information generating unit 610 generates the position and posture information indicating the information of the position and the posture of each recognized element. The position and posture information generating unit 610 sends the generated position and posture information to the planning unit 530.

The position and posture information generating unit 610 recognizes the position and the posture of each element according the following procedures, for example. For example, the position and posture information generating unit 610 first analyzes the working information to specify the article being the working target. Next, the position and posture information generating unit 610 accesses to the recognition information storing unit 510 and extracts, by taking the identification information of the article being the working target as a key, from among the information stored in the recognition information storing unit 510, at least one of the two-dimensional information, the three-dimensional information and the feature information of the article associated with the identification information. At this time, the position and posture information generating unit 610 may also extract holding information that is associated with the identification information of the article being the working target.

The recognition information storing unit 510 may not store information of some articles of the articles included in the working information. Here, in a case in which the information representing the article being the working target is not stored in the recognition information storing unit 510, the position and posture information generating unit 610 accesses to the article information storing unit 250 and extracts, by taking the identification information of the article as the key, from among the information stored in the article information storing unit 250, at least one of the two-dimensional information, the three-dimensional information and the feature information of the article associated with the identification information. At this time, the position and posture information generating unit 610 may also extract holding information associated with the identification information of the article being the working target. When the position and posture information generating unit 610 acquires information from the article information storing unit 250, the position and posture information generating unit 610 may store, in the recognition information storing unit 510, the information and the identification information of the above-described article in association with each other.

Next, the position and posture information generating unit 610 acquires, from the image capturing apparatus 151, the image capturing apparatus 152, the image capturing apparatus 154, and the image capturing apparatus 155, image data of each of the one or more supply pallets and the one or more delivery pallets arranged on the working positions of the transfer apparatus 130. The recognition unit 520 may also acquire, from the image capturing apparatus 151, the image capturing apparatus 152, the image capturing apparatus 154, and the image capturing apparatus 155, point cloud data of each of the one or more supply pallets and the one or more delivery pallets arranged on the working positions of the transfer apparatus 130. Accordingly, the position and posture information generating unit 610 can acquire the information representing the state of the surface at least one of the supply pallet and the delivery pallet.

Next, the position and posture information generating unit 610 analyzes the image data or the point cloud data of each of the one or more supply pallets by using at least one of the two-dimensional information, the three-dimensional information and the feature information of the article being the working target to recognize (i) the position and the posture of each supply pallet and (ii) the positions and the postures of at least some of the one or more articles stored in each supply pallet. Similarly, the position and posture information generating unit 610 recognizes (i) the position and the posture of the delivery pallet and (ii) the positions and the postures of at least some of the one or more articles stored in the delivery pallet.

The position and posture information generating unit 610 may also select the work 10 being the working target of the transfer apparatus 130 from among the articles stored in the one or more supply pallets. The position and posture information generating unit 610 may select the work 10 being the working target of the transfer apparatus 130 from among the articles whose positions and postures having already been recognized. Note that the process of selecting the work 10 may also be performed in the planning unit 530.

In the present embodiment, the point cloud information generating unit 620 generates the point cloud information of at least one of the supply pallet and the delivery pallet. The point cloud information generating unit 620 generates the point cloud information according to the following procedures, for example. In one embodiment, the point cloud information generating unit 620 first acquires, from the image capturing apparatus 151, the image capturing apparatus 152, the image capturing apparatus 154, and the image capturing apparatus 155, point cloud data of each of the one or more supply pallets 12 and the one or more delivery pallets 14 arranged on the working positions of the transfer apparatus 130. Next, the point cloud information generating unit 620 associates the point cloud data of each of the one or more supply pallets 12 and the one or more delivery pallets 14 with the identification information thereof to generate the point cloud information of each of the one or more supply pallets 12 and the one or more delivery pallets 14.

In another embodiment, the point cloud information generating unit 620 first acquires, from the image capturing apparatus 151, the image capturing apparatus 152, the image capturing apparatus 154, and the image capturing apparatus 155, image data of each of the one or more supply pallets 12 and the one or more delivery pallets 14 arranged on the working positions of the transfer apparatus 130. Next, the point cloud information generating unit 620 acquires the point cloud data of each of the one or more supply pallets 12 and the one or more delivery pallets 14 based on the image data of each of the one or more supply pallets 12 and the one or more delivery pallets 14 arranged on the working positions of the transfer apparatus 130. Next, the point cloud information generating unit 620 associates the point cloud data of each of the one or more supply pallets 12 and the one or more delivery pallets 14 with the identification information thereof to generate the point cloud information of each of the one or more supply pallets 12 and the one or more delivery pallets 14.

In the present embodiment, the point cloud information generating unit 620 sends the point cloud information to the planning unit 530. The point cloud information generating unit 620 may also send the point cloud information to the execution unit 540. The point cloud information generating unit 620 may also generate the point cloud information in accordance with an instruction or a request from the execution unit 540 or the concurrent processing control unit 550, and send the point cloud information to the execution unit 540.

FIG. 7 schematically shows one example of an internal configuration of the planning unit 530. In the present embodiment, the planning unit 530 includes a holding planning unit 710, an arrangement planning unit 720, an evaluation information storing unit 730, a trajectory candidate extracting unit 740, a trajectory planning unit 750 and a control information generating unit 760.

The holding planning unit 710 may be one example of a first candidate information generating unit. The arrangement planning unit 720 may be one example of a second candidate information generating unit. The trajectory candidate extracting unit 740 may be one example of the first candidate information acquiring unit, the second candidate information acquiring unit and the extracting unit. The trajectory planning unit 750 may be one example of the trajectory information generating unit, the trajectory calculating unit, the determining unit, the trajectory determining unit, and the interpolation trajectory generating unit.

In the present embodiment, the holding planning unit 710 plans a holding method of the work 10 by the picking hand 412. Specifically, the holding planning unit 710 determines the position and the posture of the picking hand 412 when the picking hand 412 holds the work 10 stored in the supply pallet 12. For example, the holding planning unit 710 extracts one or more postures (which may be referred to as holding candidates) that the picking hand 412 can take on a position (which may be referred to as a picking position) on which the picking hand 412 holds the work 10, with reference to (i) the position and posture information of the supply pallet 12, (ii) the position and posture information of the one or more articles stored in the supply pallet 12, (iii) the point cloud information of the supply pallet 12 and (iv) the holding information acquired by the recognition unit 520 from the recognition information storing unit 510 or the article management apparatus 110. The picking position may be one example of a first position.

For example, if the picking hand 412 holds, according to the holding method indicated by the holding information, an article that is of at least some of the one or more articles stored in the supply pallet 12, the holding planning unit 710 determines whether or not the picking hand 412 interferes with the supply pallet 12 and other articles. When the holding planning unit 710 determines that the picking hand 412 does not interfere with the supply pallet 12 and other articles, the holding planning unit 710 extracts the position and the posture of the picking hand 412 at that time as the holding candidate. The holding planning unit 710 repeats the above-described determining processing within a range of a constraint condition regarding the amount or duration time of calculation set by the concurrent processing control unit 550. Accordingly, the holding planning unit 710 can generate the information representing one or more holding candidates. The holding planning unit 710 sends the information representing the one or more holding candidates to the trajectory candidate extracting unit 740. The information representing the one or more holding candidates may be one example of the first candidate information.

In the present embodiment, the arrangement planning unit 720 plans an arrangement method of the work 10 by the picking hand 412. Specifically, the arrangement planning unit 720 determines the position and the posture of the picking hand 412 when the picking hand 412 arranges the work 10 on the delivery pallet 14. For example, the arrangement planning unit 720 extracts one or more postures (which may be referred to as arrangement candidates) that the picking hand 412 can take on a position (which may be referred to as an arrangement position) on which the picking hand 412 arranges the work 10, with reference to (i) the position and posture information of the delivery pallet 14, (ii) the position and posture information of the one or more articles stored in the delivery pallet 14, (iii) the point cloud information of the delivery pallet 14 and (iv) the holding information acquired by the recognition unit 520 from the recognition information storing unit 510 or the article management apparatus 110. The arrangement position may be one example of a second position.

For example, if the picking hand 412 arranges, according to the holding method indicated by the holding information, the work 10 on any position or a specific position of the delivery pallet 14, the arrangement planning unit 720 determines whether or not the picking hand 412 and the work 10 interfere with the delivery pallet 14 and other articles. When the arrangement planning unit 720 determines that the picking hand 412 and the work 10 do not interfere with the delivery pallet 14 and the other articles, the arrangement planning unit 720 extracts the position and the posture of the picking hand 412 at that time as the arrangement candidate. The arrangement planning unit 720 repeats the above-described determining processing within a range of the constraint condition regarding the amount or duration time of calculation set by the concurrent processing control unit 550. Accordingly, the arrangement planning unit 720 can generate the information representing one or more arrangement candidates. Note that the arrangement planning unit 720 may also repeat the above-described determining processing until all articles included in the user order information are arranged on the delivery pallets 14. The arrangement planning unit 720 sends the information representing the one or more arrangement candidates to the trajectory candidate extracting unit 740. The information representing the one or more arrangement candidates may be one example of the second candidate information.

In the present embodiment, the evaluation information storing unit 730 stores information (which may be referred to as evaluation information) regarding a determination reference for extracting a specific trajectory candidate from among one or more trajectory candidates during a process of extracting the trajectory candidates by the trajectory candidate extracting unit 740. As the evaluation information, (i) information for evaluating a period of time during which the picking hand 412 moves from one position to another position, (ii) information for evaluating a period of time during which the picking hand 412 moves from one position to another position with a posture, (iii) information for evaluating a period of time during which the picking hand 412 moves from one position with one posture to another position and (iv) information for evaluating a period of time during which the picking hand 412 moves from one position with one posture to another position with another posture can be illustrated. The period of time of the movement from one position to another position may be evaluated based on an actual measurement value during a trial operation, a previous operation history, empirical laws and the like, for example.

In the present embodiment, the trajectory candidate extracting unit 740 extracts a specific candidate (which may be referred to as a trajectory candidate) from among a plurality of candidates associated with the trajectories of the picking hand 412. The trajectory candidate is indicated according to an initial arrangement and a final arrangement of the picking hand 412, for example. The trajectory candidate may also be indicated according to the initial arrangement, one or more passing arrangements, and the final arrangement of the picking hand 412. Each of the initial arrangement, the one or more passing arrangements, and the final arrangement of the picking hand 412 may also include information representing the positions and postures of the picking hand 412. At least one of the initial arrangement, the one or more passing arrangements, and the final arrangement of the picking hand 412 may include information representing the positions of the picking hand 412 and also, may not include the information representing the postures of the picking hand 412.

The trajectory candidate extracting unit 740 extracts the trajectory candidates according to the following procedures, for example. The trajectory candidate extracting unit 740 first receives information representing one or more holding candidates from the holding planning unit 710. Accordingly, for example, the trajectory candidate extracting unit 740 acquires M holding candidates, where M is an integer being 0 or equal to or more than 1. Also, the trajectory candidate extracting unit 740 receives information representing the one or more arrangement candidates from the arrangement planning unit 720. Accordingly, for example, the trajectory candidate extracting unit 740 acquires N arrangement candidates, where N is an integer being 0 or equal to or more than 1.

At this time, the trajectory candidate extracting unit 740 may also acquire information indicating a position (which may be referred to as a starting position) of the picking hand 412 when the robot 140 starts working on a work or a position and a posture of the picking hand 412 when the robot 140 starts working on the work. The starting position may be one example of a third position. In one embodiment, the starting position of the picking hand 412 may be an immediately preceding arrangement position of the work. Also, the posture of the picking hand 412 on the starting position may be a posture of the picking hand 412 on the immediately preceding arrangement position of the work. In another embodiment, the position and the posture of the picking hand 412 when the robot 140 starts working on the work may also be predetermined.

Next, the trajectory candidate extracting unit 740 extracts one combination from among M×N combinations regarding M holding candidates and N arrangement candidates. The trajectory candidate extracting unit 740 sends the information indicating the extracted combination, as the trajectory candidates, to the trajectory planning unit 750.

The trajectory candidate extracting unit 740 extracts the combination being the trajectory candidate according to the following procedures, for example. First, the trajectory candidate extracting unit 740 calculates, for a combination of one of the M holding candidates and one of the N arrangement candidates, an evaluation function for a period of time during which the picking hand 412 moves from holding position and posture indicated by the selected holding candidate to arrangement position and posture indicated by the selected arrangement candidate. The trajectory candidate extracting unit 740 may calculate the above-described evaluation function with reference to the information stored in the evaluation information storing unit 730. For example, the evaluation function includes an evaluation item regarding an estimated value of a period of time that is from (i) a time when the picking hand 412 picks the work on a holding position with a posture indicated by the selected holding candidate to (ii) a time when the picking hand 412 arranges the work on an arrangement position with a posture indicated by the selected arrangement candidate.

Note that the trajectory candidate extracting unit 740 may also calculate the above-described evaluation function for a trajectory to which the starting position and posture are set as the initial arrangement, the holding position and posture are set as the passing arrangement and the arrangement position and posture are set as the final arrangement. At this time, the evaluation function may include an evaluation item regarding an estimated value of a period of time during which (i) the picking hand 412 moves from the starting position to the holding position, (ii) the picking hand 412 picks a work on the holding position with a posture indicated by the selected holding candidate and (iii) the picking hand 412 arranges the work on the arrangement position with a posture indicated by the selected arrangement candidate.

Next, the trajectory candidate extracting unit 740 repeats the process of calculating the evaluation function for each of the combinations of the holding candidates and the arrangement candidates within a range of the constraint condition regarding the amount or duration time of calculation set by the concurrent processing control unit 550. The trajectory candidate extracting unit 740 selects a combination with the highest evaluation, as the trajectory candidate, among the combinations calculated according to the evaluation function. For example, when the evaluation function is an estimated value of a period of time during which the picking hand 412 moves from the starting position to the arrangement position, if the estimated value of the combination is smaller, the evaluation for the combination is higher. Note that when the calculated evaluation function satisfies the predetermined condition, even if the above-described constraint condition is satisfied, the trajectory candidate extracting unit 740 may also cancel the process of calculating the evaluation function and select a combination of the holding candidate and the arrangement candidate at that time, as the trajectory candidate.

In the present embodiment, the trajectory planning unit 750 generates the trajectory information defining the trajectory for which the picking hand 412 picks a work on a holding position and arranges the work on an arrangement position. The trajectory planning unit 750 may also generate trajectory information defining a trajectory for which the picking hand 412 moves from the starting position to a holding position, picks a work on the holding position, and arranges the work to an arrangement position. The trajectory planning unit 750 sends the generated trajectory information to the control information generating unit 760.

The trajectory planning unit 750 generates the trajectory information according to the following procedures, for example. The trajectory planning unit 750 first receives information indicating a combination of a holding candidate and an arrangement candidate from the trajectory candidate extracting unit 740. Next, the trajectory planning unit 750 calculates a trajectory for which (i) the picking hand 412 picks, on the holding position, a work with a posture indicated by the holding candidate included in the combination extracted by the trajectory candidate extracting unit 740 and (ii) the picking hand 412 arranges, on the arrangement position, the work with a posture indicated by the arrangement candidate included in the combination extracted by the trajectory candidate extracting unit 740. At this time, the trajectory planning unit 750 may also calculate a trajectory for which (i) the picking hand 412 moves from the starting position to the holding position, (ii) the picking hand 412 picks, on the holding position, a work with a posture indicated by the holding candidate included in the combination extracted by the trajectory candidate extracting unit 740 and (iii) the picking hand 412 arranges, on the arrangement position, the work with a posture indicated by the arrangement candidate included in the combination extracted by the trajectory candidate extracting unit 740.

The trajectory planning unit 750 may calculate the above-described trajectory from the combination of the holding candidate and the arrangement candidate according to a predetermined algorithm. The trajectory planning unit 750 may also calculate the above-described trajectory from the combination of the holding candidate and the arrangement candidate such that the generated trajectory satisfies a predetermined condition.

Next, the trajectory planning unit 750 determines whether the interference is present or absent on the calculated trajectory. For example, the trajectory planning unit 750 determines whether or not another object (for example, an obstacle, another device, another element of the robot 140, and the like) interferes with the picking hand 412 when the picking hand 412 moves along the calculated trajectory. Note that during a step after the picking hand 412 picks the work 10, the trajectory planning unit 750 determines whether or not another object interferes with the picking hand 412 and the work 10.

The trajectory planning unit 750 may also divide the calculated trajectory into a plurality of zones and determine whether the interference is present or absent for each zone. The process of determining the interference for each zone may be performed concurrently or in parallel. When there is even only one zone in which the interference occurs, the trajectory planning unit 750 can determine that the interference occurs on the trajectory even if the determining processing for another zone is not ended. Accordingly, the determination time can be shortened.

The trajectory planning unit 750 determines the trajectory of the picking hand 412 based on the above-described determination result. For example, the trajectory planning unit 750 generates an interpolation trajectory of the trajectory if the trajectory planning unit 750 determines that no interference occurs on the calculated trajectory. Accordingly, an excessive acceleration can be prevented from being applied to the work 10. Also, the trajectory planning unit 750 generates the trajectory information defining the above-described interpolation trajectory. The trajectory information may be information indicating an angle of each joint of the robot arm 414 at each time.

On the other hand, when the trajectory planning unit 750 determines that the interference occurs on the calculated trajectory, the trajectory planning unit 750 accesses to the trajectory candidate extracting unit 740 to request for information indicating a combination having an evaluation that is next highest after the previous combination. As the trajectory planning unit 750 receives the information indicating a new combination from the trajectory candidate extracting unit 740, the trajectory planning unit 750 generates the trajectory information according to procedures similar to the above-described procedures. The combination having the next highest evaluation may be one example of another combination.

In a case in which the amount or duration time of calculation in the trajectory planning unit 750 satisfies the predetermined condition, the trajectory planning unit 750 suspends the process regarding the current combination. For example, the trajectory planning unit 750 determines the amount or duration time of calculation allocated to the above-described determining processing so as to generate the trajectory information within a range of the constraint condition regarding the amount or duration time of calculation set by the concurrent processing control unit 550. If the determining processing is not ended even when the determined amount or duration time of calculation is exceeded, the trajectory planning unit 750 suspends the determining processing regarding the current combination. After that, the trajectory planning unit 750 accesses to the trajectory candidate extracting unit 740 to request for information indicating a combination having an evaluation that is next highest after the current combination. As the trajectory planning unit 750 receives the information indicating a new combination from the trajectory candidate extracting unit 740, the trajectory planning unit 750 generates the trajectory information according to procedures similar to the above-described procedures.

In the present embodiment, the control information generating unit 760 generates the control information for controlling the picking hand 412 and the robot arm 414. For example, the control information generating unit 760 receives the trajectory information from the trajectory planning unit 750. For example, the control information generating unit 760 adds, to the trajectory information generated by the trajectory planning unit 750, the another information that is necessary for controlling the picking hand 412 and the robot arm 414 to generate the control information. The control information generating unit 760 may also convert the trajectory information generated by the trajectory planning unit 750 into a format in accordance with the type of the robot 140 to generate the control information. The control information generating unit 760 sends the control information to the execution unit 540.

FIG. 8 schematically shows another example of the internal configuration of the execution unit 540. In the present embodiment, the execution unit 540 includes a control information sending unit 810 and an execution monitoring unit 820. The control information sending unit 810 may be one example of the instruction sending unit. The execution monitoring unit 820 may be one example of the control device, the point cloud information acquiring unit, the abnormality detection unit and the instruction sending unit.

In the present embodiment, the control information sending unit 810 sends various control information to each unit of the distribution management system 100 to control at least some of the distribution management system 100. The control information sending unit 810 may also control the transfer apparatus 130 or the transfer control apparatus 160. For example, the control information sending unit 810 sends the control information for controlling the robot 140 to the drive control unit 430 to control the operation of the robot 140. If the control information sending unit 810 receives, from the execution monitoring unit 820, a detection signal indicating that an abnormality is detected, the control information sending unit 810 may also send an instruction for suspending the operation of the robot 140 to the robot 140.

The control information sending unit 810 may also control at least one of the image capturing apparatus 151, the image capturing apparatus 152, the image capturing apparatus 154, and the image capturing apparatus 155. For example, the control information sending unit 810 may control a position, an angle, magnification and the like of an image capturing apparatus for each image capturing apparatus.

The control information sending unit 810 may also control at least one of the recognition unit 520 and the planning unit 530. For example, if an abnormality occurs on the robot 140, the control information sending unit 810 suspends the current process and sends an instruction for starting new recognition processing and planning processing to the recognition unit 520 and the planning unit 530. The control information sending unit 810 requests, at a specific timing, the point cloud information generating unit 620 to send point cloud information of at least one of the supply pallet 12 and the delivery pallet 14. The supply pallet 12 and the delivery pallet 14 may be one example of a region including at least one of the picking hand 412 and the work 10.

As the above-described specific timing, at least one of the followings can be illustrated: (i) a timing at which the picking hand 412 picks the work 10 on the holding position, (ii) a timing at which the picking hand 412 moves the work 10 out of the supply pallet 12, (iii) a timing at which the picking hand 412 moves the work 10 in the delivery pallet 14, (iv) a timing at which the picking hand 412 arranges the work 10 on the arrangement position. Accordingly, an abnormality such as an interference and load collapse can be detected in an early stage.

In the present embodiment, the execution monitoring unit 820 monitors the operation of the robot 140, the supply pallet 12 and the delivery pallet 14. For example, the execution monitoring unit 820 receives, from the point cloud information generating unit 620, the point cloud information of at least one of the supply pallet 12 and the delivery pallet 14 at the above-described specific timing. The execution monitoring unit 820 detects the abnormality based on the above-described point cloud information. The execution monitoring unit 820 may also detect the abnormality based on (i) the above-described point cloud information and (ii) at least one selected from a group including: a measurement value of a sensor arranged on the robot 140; a control value of at least one of the picking hand 412 and the robot arm 414; and at least one of torque, a voltage value and a current value of the driving unit 420.

If the abnormality is detected, the execution monitoring unit 820 sends a detection signal indicating that the abnormality is detected to the control information sending unit 810, for example. If the abnormality is detected, the execution monitoring unit 820 may also send an instruction for suspending the operation of the robot 140 to the robot 140.

FIG. 9 schematically shows one example of information processing in the transfer control apparatus 160. According to the present embodiment, first, in a step 902 (the term “step” may be abbreviated as the letter “S”), the recognition unit 520 receives user order information or working information from the article management apparatus 110. The recognition unit 520 analyzes the user order information or the working information. For example, the recognition unit 520 determines the picking sequence of one or more articles to the transfer apparatus 130 based on the user order information or the working information. The recognition unit 520 may generate information in which the identification information of the one or more articles are arranged in the picking sequence to the transfer apparatus 130.

In S912, the recognition unit 520 performs recognition processing. In S914, the planning unit 530 performs planning processing. In S916, the execution unit 540 performs execution processing. Also, the execution unit 540 monitors the operation of the robot 140 and the supply pallet 12 and the delivery pallet 14 during a period in which the robot 140 transfers the work 10. When occurrence of the abnormality is detected, the execution unit 540 suspends the processing in S916 and starts the processing from S912 again.

Next, in S920, the execution unit 540 determines whether or not the transfer processing of all articles included in the user order information or working information is ended, for example. If the execution unit 540 determines that the transfer processing of all articles is ended (in a case of “Yes” in S920), the execution unit 540 ends the processing. On the other hand, if the execution unit 540 determines that the transfer processing of all articles is not ended (in a case of “No” in S920), the execution unit 540 performs, on a next article, processing from S912 to S920.

FIG. 10 schematically shows one example of the recognition processing in S912. In the present embodiment, (i) the process of generating the point cloud information and the position and posture information representing the supply pallet 12 and (ii) the process of generating the point cloud information and the position and posture information representing the delivery pallet 14 are performed concurrently or in parallel. In the present embodiment, the process of generating the point cloud information and the process of generating the position and posture information are performed concurrently or in parallel.

According to the present embodiment, in S1022, the position and posture information generating unit 610 acquires the image data of the supply pallet 11 from the image capturing apparatus 151 and acquires the image data of the supply pallet 12 from the image capturing apparatus 152. Also, the point cloud information generating unit 620 acquires the point cloud data of the supply pallet 11 from the image capturing apparatus 151 and acquires the point cloud data of the supply pallet 12 from the image capturing apparatus 152. The point cloud information generating unit 620 may also generate the point cloud data of the supply pallet 11 based on the image data of the supply pallet 11. The point cloud information generating unit 620 may also generate the point cloud data of the supply pallet 12 based on the image data of the supply pallet 12.

Next, in S1024, the point cloud information generating unit 620 generates the point cloud information of the supply pallet 11 and the point cloud information of the supply pallet 12. In S1026, the position and posture information generating unit 610 generates the position and posture information of the supply pallet 11 and the position and posture information of the article stored in the supply pallet 11. Also, the position and posture information generating unit 610 generates the position and posture information of the supply pallet 12 and the position and posture information of the article stored in the supply pallet 12. In the present embodiment, S1024 and S1026 are performed concurrently or in parallel. Note that during a period in which S1026 is performed for one time, S1024 may be performed for multiple times.

On the other hand, in S1042, the position and posture information generating unit 610 acquires the image data of the delivery pallet 14 from the image capturing apparatus 154 and acquires the image data of the delivery pallet 15 from the image capturing apparatus 155. Also, the point cloud information generating unit 620 acquires the point cloud data of the delivery pallet 14 from the image capturing apparatus 154 and acquires the point cloud data of the delivery pallet 15 from the image capturing apparatus 155. The point cloud information generating unit 620 may also generate the point cloud data of the delivery pallet 14 based on the image data of the delivery pallet 14. The point cloud information generating unit 620 may also generate the point cloud data of the delivery pallet 15 based on the image data of the delivery pallet 15. In the present embodiment, S1022 and S1042 are performed concurrently or in parallel.

Next, in S1044, the point cloud information generating unit 620 generates the point cloud information of the delivery pallet 14 and the point cloud information of the delivery pallet 15. In S1046, the position and posture information generating unit 610 generates the position and posture information of the delivery pallet 14 and the position and posture information of the article stored in the delivery pallet 14. Also, the position and posture information generating unit 610 generates the position and posture information of the delivery pallet 15 and the position and posture information of the article stored in the delivery pallet 15. In the present embodiment, S1044 and S1046 are performed concurrently or in parallel. Note that during a period in which S1046 is performed for one time, S1044 may also be performed for multiple times.

FIG. 11 schematically shows one example of the planning processing in S914. In the present embodiment, the process of extracting the holding candidate in the holding planning unit 710 and the process of extracting the arrangement candidate in the arrangement planning unit 720 are performed concurrently or in parallel. According to the present embodiment, in S1120, the holding planning unit 710 extracts the holding candidate. Also, in S1140, the arrangement planning unit 720 extracts the arrangement candidate. In the present embodiment, S1120 and S1140 are performed concurrently or in parallel. After that, in S1160, the trajectory candidate extracting unit 740 extracts the trajectory candidate and the trajectory planning unit 750 determines the trajectory. Also, in S1180, an interpolation trajectory of the determined trajectory is generated.

FIG. 12 schematically shows one example of the trajectory planning processing in S1160. In the present embodiment, the processes in which the trajectory candidate extracting unit 740 calculates the evaluation functions for at least some of M×N combinations regarding M holding candidates and N arrangement candidates are performed concurrently or in parallel. According to the present embodiment, in S1202, the trajectory candidate extracting unit 740 calculates the evaluation function for each of the M×N combinations regarding the M holding candidates and the N arrangement candidates. At this time, the trajectory candidate extracting unit 740 performs a process of calculating the evaluation function for each of the M×N combinations concurrently or in parallel. After that, in S1204, the trajectory candidate extracting unit 740 extracts one combination from among the M×N combinations based on the evaluation function of each combination.

Next, in S1206, the trajectory planning unit 750 calculates a trajectory for which (i) the picking hand 412 picks, on a holding position, a work with a posture indicated by the holding candidate included in the combination extracted by the trajectory candidate extracting unit 740 and (ii) the picking hand 412 arranges, on an arrangement position, the work with a posture indicated by the arrangement candidate included in the combination extracted by the trajectory candidate extracting unit 740.

In S1208, the trajectory planning unit 750 determines whether the interference is present or absent on the calculated trajectory. In the present embodiment, the trajectory planning unit 750 divides the calculated trajectory into a plurality of zones to determine whether the interference is present or absent for each zone. If it is determined that no interference occurs on the trajectory in all zones (in a case of “Yes” in S1208), the trajectory planning unit 750 ends the processing. Note that even in this case, the processing from S1204 to S1208 may also be repeated within a range of the constraint condition regarding the amount or duration time of calculation set by the concurrent processing control unit 550. On the other hand, if it is determined that an interference occurs on the trajectory in at least some zones (in a case of “No” in S1208), the processing from S1204 to S1208 is repeated.

FIG. 13 schematically shows one example of the interference checking processing in S1208. In the present embodiment, the trajectory planning unit 750 divides the trajectory calculated in S1206 into a plurality of zones to determine whether an interference is present or absent for each zone. Also, the trajectory planning unit 750 performs the determining processing in at least two zones concurrently or in parallel. According to the present embodiment, in S1302, the trajectory planning unit 750 determines whether the interference is present or absent on the trajectory calculated in S1206. The trajectory planning unit 750 divides the trajectory calculated in S1206 into a plurality of zones to determine whether the interference is present or absent for each zone. The trajectory planning unit 750 performs the determining processing in all zones concurrently or in parallel, for example.

In the determining processing for each zone, if the duration time or amount of calculation exceeds a predetermined threshold before the determining processing is ended (in a case of “Yes” in S1304), the trajectory planning unit 750 cancels the determining processing in all zones and returns to the processing in S1204. On the other hand, if the determining processing is ended before the duration time or amount of calculation exceeds the predetermined threshold and when it is determined that an interference occurs in one zone (in a case of “Yes” in S1306), the trajectory planning unit 750 cancels the determining processing in all zones and returns to the processing in S1204. On the other hand, if the determining processing is ended before the duration time or amount of calculation exceeds the predetermined threshold and when it is determined that an interference does not occur in the zone (in a case of “No” in S1306), the determining processing in the zone is ended.

[Summary of Concurrent Processing or Parallel Processing]

The concurrent processing or parallel processing between works or between orders in the transfer control apparatus 160 is described by using FIG. 14, FIG. 15 and FIG. 16. FIG. 14 schematically shows one example of the data table 1400. In the present embodiment, a case in which the transfer control apparatus 160 manages transfer processing of a plurality of works corresponding to a plurality of pieces of user order information based on the data table 1400 is described. FIG. 15 schematically shows one example of the concurrent processing or parallel processing between works. FIG. 16 schematically shows one example of the concurrent processing or parallel processing between works.

As shown in FIG. 14, in the present embodiment, the data table 1400 stores a work ID 1410, an order ID 1420 and contents 1430 of an order in association with each other. The contents 1430 of the order include an article ID 1432 for identifying an article to be a transfer target, a supply pallet ID 1434 for identifying a supply pallet, and a delivery pallet ID 1436 for identifying a delivery pallet, for example.

The data table 1400 includes information indicating the picking sequence (which may be referred to as an execution sequence) of one or more works on the transfer apparatus 130. The data table 1400 is generated by the transfer apparatus management unit 230 or the transfer control apparatus 160, for example. The data table 1400 may be one example of the working information indicating the working contents of the transfer apparatus 130. At least one of the data table 1400 and each record configuring the data table 1400 may be one example of the order information that includes instructions to move the one or more works from at least one of the one or more supply pallets to at least one of the one or more delivery pallets.

In the embodiment of FIG. 14, the user A orders one product X (product ID: I_0001), one product Y (product ID 1432: I_0002) and one product Z (product ID 1432: I_0003), and the user B orders one product X and two products Y, for example. An order ID of the order of the user A is OD_0001 and an order ID of the order of the user B is OD_0002. Also, a supply pallet having a supply pallet ID 1434 being S_0001 (which may be referred to as a pallet S1) is arranged on a position of the supply pallet 11 of FIG. 1, (ii) a supply pallet having the supply pallet ID 1434 being S_0002 (which may be referred to as a pallet S2) is arranged on a position of the supply pallet 12, (iii) a delivery pallet having a delivery pallet ID 1436 being D_0001 (which may be referred to as a pallet D1) is arranged on a position of a delivery pallet 14, and (iv) a delivery pallet having the delivery pallet ID 1436 being D_0002 (which may be referred to as a pallet D2) is arranged on a position of the delivery pallet 15.

In the above-described case, the data table 1400 is generated in the following procedures, for example. According to the present embodiment, the order information acquiring unit 210 first receives user order information indicating the contents of the order of the user A and user order information indicating the contents of the order of the user B. The order information acquiring unit 210 sends the received user order information to the transfer apparatus management unit 230.

Next, the transfer apparatus management unit 230 determines an execution sequence of a plurality of works. For example, the transfer apparatus management unit 230 arranges a sequence of a plurality of works including one or more works corresponding to the order of the user A (which are the works having the work ID 1410 respectively being W_0001, W_0005 and W_0006 and may be respectively referred to as a work W1, work W5 and work W6) and one or more works corresponding to the order of the user B (which are the works having the work ID 1410 respectively being W_0002, W_0003 and W_0004 and may be respectively referred to as a work W2, work W3 and work W4) to create the data table 1400 that prescribes the execution sequence of the plurality of works.

In one embodiment, the transfer apparatus management unit 230 creates the data table 1400 such that the amount or duration time of calculation in at least one of the recognition unit 520 and the planning unit 530 becomes small. For example, the transfer apparatus management unit 230 determines the execution sequence of the plurality of works such that the planning unit 530 can generate, by using an operation result in the process of generating the trajectory information representing one work, trajectory information representing another work.

In another embodiment, the transfer apparatus management unit 230 creates the data table 1400 such that the working time or waiting time of the robot 140 becomes small. For example, in the embodiment of FIG. 14, after the transfer processing of the work W2 is ended, during a period in which the transfer processing of the work W3, the work W4 and the work W5 are performed, the pallet S1 is moved out of the working position of the transfer apparatus 130 and a supply pallet having the supply pallet ID 1434 being S_0003 (which may be referred to as a pallet S3) moves in the working position of the transfer apparatus 130. Because sufficient time for moving the supply pallets in and out of the working position is secured, the robot 140 can start the transfer processing of a work having the work ID 1410 being W_0006 (which may be referred to as a work W6) without waiting for the pallet S3 moved in the working position.

To simplify the description, in the present embodiment, a case of determining an execution sequence of a plurality of works corresponding to two orders is described. However, people skilled in this art who are in contact with the description of the specification of the present application can understand that even if the number of the orders is three or more, the execution sequence of the plurality of works corresponding to the three or more orders can be determined according to a similar procedure to that of the case in which the number of the orders is two. Note that as described above, the transfer control apparatus 160 may also generate the data table 1400 based on one or more pieces of user order information.

[Control of Transfer Apparatus 130 by Using Data Table 1400]

The transfer control apparatus 160 or each unit thereof performs the transfer processing of one or more works according to the sequence and the contents indicated in the data table 1400. For example, first, the recognition unit 520 reads records in the data table 1400 sequentially. Then, in accordance with one record acquired by the recognition unit 520, the planning unit 530 performs the process of generating trajectory information representing a work indicated in the one record. Similarly, in accordance with another record acquired by the recognition unit 520, the planning unit 530 performs the process of generating trajectory information representing a work indicated in the other record.

In one embodiment, the data table 1400 includes (i) one or more records that instruct to move one or more works from at least one of one or more supply pallets to one delivery pallet, and (ii) one or more records that instruct to move one or more works from at least one of one or more supply containers to another delivery pallet. The above-described records of (i) may be one example of the first order information. The above-described records of (ii) may be one example of the second order information. One delivery pallet may be one example of a first arrangement container included in one or more arrangement containers. The other delivery pallet may be one example of a second arrangement container included in the one or more arrangement containers and different from the first arrangement container.

In another embodiment, the data table 1400 includes (i) one or more records that instruct to move one or more works from one supply pallet to at least one of one or more arrangement pallets, and (ii) one or more records that instruct to move one or more works from another supply pallet to at least one of the one or more arrangement pallets. The above-described records of (i) may be one example of the first order information. The above-described records of (ii) may be one example of the second order information. The one supply pallet may be one example of a first supply container included in the one or more supply containers. The other supply pallet may be one example of a second supply container different from the first supply container and included in the one or more supply containers and different from the first supply container.

In still another embodiment, the data table 1400 includes (i) a record that includes instructions to move one work from one supply pallet to one arrangement pallet and (ii) a record that includes instructions to move another work from another supply pallet to another arrangement pallet. The above-described record of (i) may be one example of the first order information. The above-described record of (ii) may be one example of the second order information. The one supply pallet may be one example of the first supply container included in the one or more supply containers. The other supply pallet may be one example of the second supply container different from the first supply container and included in the one or more supply containers and different from the first supply container. The one arrangement pallet may be one example of the first arrangement container included in the one or more arrangement containers. The other arrangement pallet may be one example of the second arrangement container included in the one or more arrangement containers and different from the first arrangement container.

As shown in FIG. 15, according to the present embodiment, the planning unit 530 performs the planning processing of the work W2 and the recognition processing of the work W3 after performing the planning processing of the work W1 and before the execution processing of the work W1 is completed. Accordingly, during the period in which the execution processing of the work W1 is performed, the planning processing of the work W2 is performed and the recognition processing of the work having the work W3 is performed.

The amount or duration time of calculation of the planning processing of the work W2 is adjusted such that the planning processing is ended before the execution processing of the work W1 is completed, for example. More specifically, the concurrent processing control unit 550 adjusts the amount or duration time of calculation of the planning unit 530 according to the following procedures.

For example, the concurrent processing control unit 550 estimates a period of time that is from the time when the robot 140 receives control information representing a work W1 to the time when the operation of the robot 140 on the work W1 is ended. The concurrent processing control unit 550 may also estimate the above-described period of time based on the trajectory information of the work having the work W1. The above-described period of time may be one example of the constraint condition. The concurrent processing control unit 550 sends the above-described estimation result to the planning unit 530.

On the other hand, the planning unit 530 adjusts the amount of calculation of the work planning processing such that the planning processing of the work W2 is completed before the period of time estimated by the concurrent processing control unit 550 is expired or elapsed. For example, the amount of calculation of the planning processing is adjusted by adjusting a value of the threshold in S1304.

As shown in FIG. 16, in the recognition processing of the work having the work W1, the recognition processing of the pallet S1 and the recognition processing of the pallet D1 are performed concurrently or in parallel. Also, in the recognition processing of each supply pallet, the process of generating the point cloud information and the process of generating the position and posture information are performed concurrently or in parallel. Similarly, in the recognition processing of each delivery pallet, the process of generating the point cloud information and the process of generating the position and posture information are performed concurrently or in parallel.

In the present embodiment, the process of generating the point cloud information is performed for multiple times during the period in which the recognition processing is performed for one time. Accordingly, the recognition unit 520 can provide the point cloud information to another element (for example, the execution unit 540) at a proper timing. Also, waiting time in another element until the point cloud information is provided is reduced.

[Reuse of Data in Transfer Processing of Different Works]

The transfer control apparatus 160 may generate the trajectory information representing another work by using an operation result in the process of generating the trajectory information representing one work. Accordingly, for example, the amount and duration time of calculation can be significantly reduced in a case of transferring a plurality of works from a single supply pallet to a plurality of delivery pallets on the working position of the transfer apparatus 130, or in a case of transferring a plurality of works from a plurality of supply pallets to a single delivery pallet. For example, the planning unit 530 may use, in the process of generating the trajectory information of the work W2, the information acquired or generated in the process of generating the trajectory information of work W1.

As shown in FIG. 15 and FIG. 16, the planning unit 530 performs the planning processing of the work W2 after performing the planning processing of the work W1 and before the movement of the work W1 is completed, for example. At this time, the planning unit 530 may use, in the planning processing of the work W2, the information acquired or generated in the planning processing of the work W1. The information acquired or generated in the planning processing of the work W1 may be one example of the operation result of the planning processing of the work W1.

For example, the planning unit 530 acquires the position and posture information of the pallet S1, the position and posture information of the article stored in the pallet S1 and the point cloud information of the pallet S1 in the process of generating the trajectory information of the work W1. The planning unit 530 acquires the position and posture information of the pallet D1, the position and posture information of the article stored in the pallet D1 and the point cloud information of the pallet D1. Also, the planning unit 530 generates the holding candidate in the pallet S1 and the information representing the arrangement candidate of the pallet D1. In addition, the planning unit 530 grasps the information representing the picking position and posture of the work W1 and the information representing the arrangement position of posture of the work W1.

For example, the planning unit 530 generates, in the planning processing of the work W2, the trajectory information representing the work W2 by using the information representing a state of a surface of the pallet S1 and the pallet D1, the information acquired or generated before the movement of the work W1 is completed. Similarly, the planning unit 530 may generate, in the planning processing of the work W5, the trajectory information representing the work W5 by using (i) the information representing the state of the surface of the pallet S2 acquired before the movement of the work W4 is completed and (ii) the information representing the state of the surface of the pallet D1 acquired before the movement of the work W1 is completed, or after the movement of the work W1 is completed and before the movement of W4 is completed. As the above-described information representing the state of the surface, image data, point cloud data, point cloud information, position and posture information and the like can be illustrated.

First Embodiment of Reuse of Data

In FIG. 14, for example, if comparing contents 1430 of the order of the work W1 with contents 1430 of the order of the work W2, it can be known that the same type of product is picked up from the same supply pallet. Here, for example, the planning unit 530 reuses information representing a supply pallet or an article stored in the supply pallet between the planning processing of the work W1 and the planning processing of the work W2.

In one embodiment, the holding planning unit 710 of the planning unit 530 extracts the holding candidate of the work W2 by using at least one of the position and posture information and the point cloud information acquired or generated in at least one of the recognition processing, the planning processing and the execution processing of the work W1. For example, in the planning processing of the work W2, the holding planning unit 710 extracts, as the holding candidate of the work W2, “a holding candidate that is not selected by the trajectory candidate extracting unit 740 among the holding candidates extracted by the arrangement planning unit 720 in the planning processing of the work W1”. Also, the trajectory candidate extracting unit 740 and the trajectory planning unit 750 may set, as a starting position in the planning processing of the work W2, a position and a posture of an arrangement candidate that is selected by the trajectory candidate extracting unit 740 in the planning processing of the work W1. The holding candidate and the arrangement candidate may be one example of the position and posture information. At least one of the above-described position and posture information and point cloud information may be one example of the information acquired by the container information acquiring unit before the movement of the work is completed.

Second Embodiment of Reuse of Data

In another embodiment, the planning unit 530 first acquires or generates, in the planning processing of the work W1, the position and posture information of the work W1 on the pallet S1 that is before the robot 140 moves the work W1. The position and posture information of the work W1 on the pallet S1 may also be the position and posture information acquired by the holding planning unit 710 from the recognition unit 520, and may also be the position and posture information of the holding candidate extracted by the trajectory candidate extracting unit 740. Next, the planning unit 530 generates the trajectory information representing the work W2 by using (i) the information representing the state of the surface of at least one of the pallet S1 and the pallet D1, the information acquired or generated before the movement of the work W1 is completed, and (ii) the position and posture information of the work W1 in the above-described pallet S1 in the planning processing of the work W2.

According to the present embodiment, in the planning processing of the work W2, the holding planning unit 710 extracts the holding candidate of the work W2 by using (i) at least one of the position and posture information and the point cloud information acquired in at least one of the recognition processing, the planning processing and the execution processing of the work W1, and (ii) the position and posture information of the work W1 in the pallet S1. Accordingly, the holding planning unit 710 can extract the holding candidate of the work W2 by taking the state of the pallet S1 out of which the work W1 is taken into account.

For example, in the planning processing of the work W2, the holding planning unit 710 extracts the holding candidate of the work W2 by using (i) the point cloud information of the pallet S1 acquired between the planning processing of the work W1 or the execution processing of the work W1 or the point cloud information of the pallet S1 acquired in the recognition processing of the work W2, and (ii) the position and posture information of the holding candidate selected by the trajectory candidate extracting unit 740 in the planning processing of the work W1. The holding planning unit 710 may extract the holding candidate of the work W2 by further (iii) using the position and posture information of the pallet S1 acquired or generated in the planning processing of the work W1 and the position and posture information of the article stored in the pallet S1.

Third Embodiment Reuse of Data

In still another embodiment, the planning unit 530 acquires or generates the position and posture information of the work W1 on the pallet D1 that is after the robot 140 moves the work W1 in the planning processing of the work W1. The position and posture information of the work W1 on the pallet D1 may also be the position and posture information acquired by the arrangement planning unit 720 from the recognition unit 520, or may also be the position and posture information of the arrangement candidate extracted by the trajectory candidate extracting unit 740. After that, the planning unit 530 generates the trajectory information representing the work W2 by using (i) the information representing the state of the surface of at least one of the pallet S1 and the pallet D1 acquired before the movement of the work W1 is completed, and (ii) the position and posture information of the work W1 on the above-described pallet D1 in the planning processing of the work W2. People skilled in this art who are in contact with the description for the second embodiment of the reuse of data can understand that in the present embodiment as well, the trajectory information representing the work W2 can be generated, similar to the case of the second embodiment of the reuse of data.

Fourth Embodiment of Reuse of Data

According to still another embodiment, in the planning processing of the work W2, for example, the trajectory planning unit 750 determines whether an interference is present or absent in at least some zones of the trajectory calculated from the trajectory candidates extracted by the trajectory candidate extracting unit 740, by using at least one of the position and posture information and the point cloud information acquired in at least one of the recognition processing, the planning processing and the execution processing of the work W1. At least one of the position and posture information and the point cloud information acquired in at least one of the recognition processing, the planning processing and the execution processing of the work W1 may be one example of the information acquired by the container information acquiring unit before the movement of the work is completed.

While the embodiments of the present invention have been described, the technical scope of the invention is not limited to the above described embodiments. It is apparent to persons skilled in the art that various alterations and improvements can be added to the above-described embodiments. Also, the matters described with reference to a specific embodiment can be applied to another embodiment as long as such application does not cause a technical contradiction. Also, each component may also have a similar feature to another component that has the same name but has a different reference sign. It is also apparent from the scope of the claims that the embodiments added with such alterations or improvements can be included in the technical scope of the invention.

The operations, procedures, steps, and stages of each process performed by an apparatus, system, program, and method shown in the claims, embodiments, or diagrams can be performed in any order as long as the order is not indicated by “prior to,” “before,” or the like and as long as the output from a previous process is not used in a later process. Even if the process flow is described using phrases such as “first” or “next” in the claims, embodiments, or diagrams, it does not necessarily mean that the process must be performed in this order.

For example, in the specification of the present application, the following matters are described.

[Item 1-1]

A control device for controlling a picking apparatus that arranges, on a second position, a work that is picked on a first position by using a picking hand, the second position being different from the above-described first position, the control device including:

a first candidate information acquiring unit for acquiring first candidate information indicating each of one or more postures that the above-described picking hand takes on the above-described first position;

a second candidate information acquiring unit for acquiring second candidate information indicating each of one or more postures that the above-described picking hand takes on the above-described second position;

an extracting unit for extracting one combination from among one or more combinations regarding one or more postures indicated by the above-described first candidate information and one or more postures indicated by the above-described second candidate information;

a trajectory calculating unit for calculating a trajectory for which (i) the above-described picking hand picks, on the above-described first position, the above-described work with a posture associated with the above-described first position included in the above-described one combination and (ii) the above-described picking hand arranges, on the above-described second position, the above-described work with a posture associated with the above-described second position included in the above-described one combination; and

a determining unit for determining whether an interference is present or absent on the trajectory calculated by the above-described trajectory calculating unit.

[Item 1-2]

The control device according to Item 1-1, wherein

when the above-described determining unit determines that the interference occurs on the above-described trajectory,

the above-described extracting unit extracts another combination from among the above-described one or more combinations,

the above-described trajectory calculating unit calculates a trajectory for which (i) the above-described picking hand picks, on the above-described first position, the above-described work with a posture associated with the above-described first position included in the above-described other combination and (ii) the above-described picking hand arranges, on the above-described second position, the above-described work with a posture associated with the above-described second position included in the above-described other combination, and

the above-described determining unit determines whether the interference is present or absent on the trajectory being calculated by the above-described trajectory calculating unit and being associated with the above-described other combination.

[Item 1-3]

The control device according to Item 1-1, wherein

when an amount or duration time of calculation in at least one of the above-described extracting unit, the above-described trajectory calculating unit and the above-described determining unit satisfies a predetermined condition,

the above-described extracting unit extracts another combination from among the above-described one or more combinations,

the above-described trajectory calculating unit calculates a trajectory for which (i) the above-described picking hand picks, on the above-described first position, the above-described work with a posture associated with the above-described first position included in the above-described other combination and (ii) the above-described picking hand arranges, on the above-described second position, the above-described work with a posture associated with the above-described second position included in the above-described other combination, and

the above-described determining unit determines whether the interference is present or absent on the trajectory being calculated by the above-described trajectory calculating unit and being associated with the above-described other combination.

[Item 1-4]

The control device according to any one of Items 1-1 to 1-3, wherein the above-described trajectory calculating unit calculates a trajectory for which (i) the above-described picking hand moves from a third position that is a position of the above-described picking hand when the above-described picking apparatus starts working on the above-described work to the above-described first position, (ii) the above-described picking hand picks, on the above-described first position, the above-described work with a posture associated with the above-described first position included in the combination extracted by the above-described extracting unit and (iii) the above-described picking hand arranges, on the above-described second position, the above-described work with a posture associated with the above-described second position included in the combination extracted by the above-described extracting unit.

[Item 1-5]

The control device according to any one of Items 1-1 to 1-4, wherein

the above-described extracting unit calculates an evaluation function for at least some of the above-described one or more combinations, and

selects the above-described one combination based on the above-described evaluation function.

[Item 1-6]

The control device according to Item 1-5, wherein

the above-described evaluation function includes an evaluation item regarding an estimated value of a period of time that is from (i) a time when the above-described picking hand picks, on the above-described first position, the above-described work with the posture associated with the above-described first position included in a combination that is to be an evaluation target by the above-described evaluation function to (ii) a time when the above-described picking hand arranges, on the above-described second position, the above-described work with the posture associated with the above-described second position included in the combination that is to be the evaluation target by the above-described evaluation function.

[Item 1-7]

The control device according to Item 1-5, wherein

the above-described evaluation function includes an evaluation item regarding an estimated value of a period of time that is from (i) a time when the above-described picking hand moves from a third position that is a position of the above-described picking hand when the above-described picking apparatus starts working on the above-described work to the above-described first position, (ii) the above-described picking hand picks, on the above-described first position, the above-described work with a posture associated with the above-described first position included in the combination that is to be an evaluation target by the above-described evaluation function, to (iii) a time when the above-described picking hand arranges, on the above-described second position, the above-described work with a posture associated with the above-described second position included in the combination that is to be the evaluation target by the above-described evaluation function.

[Item 1-8]

The control device according to any one of Items 1-1 to 1-7, wherein

the above-described determining unit divides the trajectory calculated by the above-described trajectory calculating unit into a plurality of zones,

determines whether an interference is present or absent for each zone.

[Item 1-9]

The control device according to any one of Items 1-1 to 1-8, further including a trajectory determining unit for determining a trajectory of the above-described picking hand based on a determination result of the above-described determining unit.

[Item 1-10]

The control device according to Item 1-9, wherein

the above-described trajectory determining unit has an interpolation trajectory generating unit for generating an interpolation trajectory of the trajectory calculated by the above-described trajectory calculating unit.

[Item 1-11]

A picking system including

the control device according to any one of Items 1-1 to 1-10; and

the above-described picking apparatus.

[Item 1-12]

A distribution system including

the control device according to any one of Items 1-1 to 1-10;

the above-described picking apparatus; and

a conveying apparatus to convey the above-described work.

[Item 1-13]

A tangible, non-transitory computer-readable medium having processor instructions stored thereon that, when executed cause the processor to perform operations for controlling a picking apparatus that moves, by using a picking hand, one or more works from one or more supply containers to one or more arrangement containers, the operations comprising:

generating trajectory information that defines a trajectory for which the picking hand picks one of the one or more works stored in one supply container included in the one or more supply containers and arranges the work in one arrangement container included in the one or more arrangement containers, wherein

the generating trajectory information includes generating, after performing a process of generating trajectory information representing a first work included in the one or more works and before a movement of the first work is completed, trajectory information representing a second work, included in the one or more works and different from the first work, by using an operation result in the process of generating the trajectory information representing the first work.

[Item 1-14]

A control method for controlling a picking apparatus that arranges, on a second position, a work picked on a first position by using a picking hand, the second position being different from the above-described first position, the control method including: a first candidate information acquiring step of acquiring first candidate information indicating each of one or more postures that the above-described picking hand takes on the above-described first position;

a second candidate information acquiring step of acquiring second candidate information indicating each of one or more postures that the above-described picking hand takes on the above-described second position;

an extracting step of extracting one combination from among one or more combinations regarding one or more postures indicated by the above-described first candidate information and one or more postures indicated by the above-described second candidate information;

a trajectory calculating step of calculating a trajectory for which (i) the above-described picking hand picks, on the above-described first position, the above-described work with a posture associated with the above-described first position included in the above-described one combination and (ii) the above-described picking hand arranges, on the above-described second position, the above-described work with a posture associated with the above-described second position included in the above-described one combination; and

a determining step of determining whether an interference is present or absent on the trajectory calculated in the above-described trajectory calculating step.

[Item 1-15]

A method of producing a container in which a work is arranged by using a picking apparatus which arranges, on a second position, the above-described work picked on a first position by using a picking hand, the second position being different from the above-described first position, the method of producing the container including:

a step of setting a specific position of the above-described container to the above-described second position;

a first candidate information acquiring step of acquiring first candidate information indicating each of one or more postures that picking hand takes on the above-described first position;

a second candidate information acquiring step of acquiring second candidate information indicating each of one or more postures that the above-described picking hand takes on the above-described second position;

an extracting step of extracting one combination from among one or more combinations regarding one or more postures indicated by the above-described first candidate information and one or more postures indicated by the above-described second candidate information;

a trajectory calculating step of calculating a trajectory for which (i) the above-described picking hand picks, on the above-described first position, the above-described work with a posture associated with the above-described first position included in the above-described one combination and (ii) the above-described picking hand arranges, on the above-described second position, the above-described work with a posture associated with the above-described second position included in the above-described one combination; and

a determining step of determining whether an interference is present or absent on the trajectory calculated in the above-described calculating the trajectory.

[Item 2-1]

A control device for controlling a picking apparatus that arranges, on a second position of a second container, a work picked on a first position of a first container by using a picking hand, the second container being different from the above-described first container, the control device including:

a trajectory information generating unit for generating trajectory information defining a trajectory for which the above-described picking hand picks the above-described work on the above-described first position and arranges the work on the above-described second position;

an execution control unit for operating the above-described picking apparatus based on the trajectory information generated by the above-described trajectory information generating unit; and

an execution time estimating unit for estimating a period of time that is from the time when the above-described picking apparatus receives an instruction for starting an operation on the above-described work to the time when the operation of the above-described picking apparatus on the above-described work is ended, wherein

the above-described trajectory information generating unit adjusts an amount of calculation based on an estimation result of the above-described execution time estimating unit.

[Item 2-2]

The control device according to Item 2-1, wherein

after the above-described trajectory information generating unit performs a process of generating trajectory information representing a first work and before a movement of the above-described first work is completed, the above-described trajectory information generating unit performs a process of generating trajectory information representing a second work being different from the above-described first work,

the above-described execution time estimating unit estimates, based on the trajectory information being generated by the above-described trajectory information generating unit and regarding the above-described first work, a first period of time that is from the time when the above-described picking apparatus receives an instruction for starting an operation on the above-described first work to the time when the operation of the above-described picking apparatus on the above-described first work is ended, and

before the above-described first period of time estimated by the above-described execution time estimating unit is expired or elapsed, the above-described trajectory information generating unit adjusts an amount of calculation of process of generating trajectory information representing the above-described second work such that process of generating trajectory information representing the above-described second work is completed.

[Item 2-3]

The control device according to Item 2-2, wherein

the above-described trajectory information generating unit generates the trajectory information representing the above-described second work by using an operation result in the process of generating the trajectory information representing the above-described first work.

[Item 2-4]

The control device according to any one of Items 2-1 to 2-3, wherein

the above-described trajectory information generating unit includes:

a first candidate information acquiring unit for acquiring first candidate information indicating each of one or more postures that picking hand takes on the above-described first position;

a second candidate information acquiring unit for acquiring second candidate information indicating each of one or more postures that the above-described picking hand takes on the above-described second position;

an extracting unit for extracting one combination from among one or more combinations regarding one or more postures indicated by the above-described first candidate information and one or more postures indicated by the above-described second candidate information;

a trajectory calculating unit for calculating a trajectory for which (i) the above-described picking hand picks, on the above-described first position, the above-described work with a posture associated with the above-described first position included in the above-described one combination and (ii) the above-described picking hand arranges, on the above-described second position, the above-described work with a posture associated with the above-described second position included in the above-described one combination; and

a determining unit for determining whether an interference is present or absent on the trajectory calculated by the above-described trajectory calculating unit.

[Item 2-5]

The control device according to Item 2-4, wherein

when the above-described determining unit determines that the interference occurs on the above-described trajectory,

the above-described extracting unit extracts another combination from among the above-described one or more combinations,

the above-described trajectory calculating unit calculates a trajectory for which (i) the above-described picking hand picks, on the above-described first position, the above-described work with a posture associated with the above-described first position included in the above-described other combination and (ii) above-described picking hand arranges, on the above-described second position, the above-described work with a posture associated with the above-described second position included in the above-described other combination, and

the above-described determining unit determines whether the interference is present or absent on the trajectory being calculated by the above-described trajectory calculating unit and being associated with the above-described other combination.

[Item 2-6]

The control device according to Item 2-4, wherein

when an amount or duration time of calculation in at least one of the above-described extracting unit, the above-described trajectory calculating unit and the above-described determining unit satisfies a predetermined condition,

the above-described extracting unit extracts another combination from among the above-described one or more combinations,

the above-described trajectory calculating unit calculates a trajectory for which (i) the above-described picking hand picks, on the above-described first position, the above-described work with a posture associated with the above-described first position included in the above-described other combination and (ii) the above-described picking hand arranges, on the above-described second position, the above-described work with a posture associated with the above-described second position included in the above-described other combination, and

the above-described determining unit determines whether the interference is present or absent on the trajectory being calculated by the above-described trajectory calculating unit and being associated with the above-described other combination.

[Item 2-7]

The control device according to any one of Items 2-4 to 2-6, further including a constraint condition setting unit for determining a constraint condition regarding an amount or duration time of calculation of processing in at least one of the above-described extracting unit, the above-described trajectory calculating unit and the above-described determining unit based on an estimation result of the above-described execution time estimating unit, wherein

the above-described at least one of the above-described extracting unit, the above-described trajectory calculating unit and the above-described determining unit adjusts an amount or duration time of calculation based on the constraint condition set by the above-described constraint condition setting unit.

[Item 2-8]

The control device according to Item 2-7, wherein

the above-described constraint condition is at least one of (i) the maximum value of the amount or duration time of calculation that is until a progress of the processing reaches a preset milestone and (ii) the maximum value of the amount or duration time of calculation that is until the processing is completed.

[Item 2-9]

The control device according to any one of Items 2-4 to 2-8, wherein

the above-described extracting unit calculates an evaluation function for at least some of the above-described one or more combinations, and

selects the above-described one combination based on the above-described evaluation function.

[Item 2-10]

The control device according to any one of Items 2-4 to 2-9, wherein

the above-described determining unit divides the trajectory calculated by the above-described trajectory calculating unit into a plurality of zones, and

determines whether an interference is present or absent for each zone.

[Item 2-11]

The control device according to any one of Items 2-4 to 2-10, wherein

the above-described trajectory information generating unit further has a trajectory determining unit for determining a trajectory of the above-described picking hand based on the determination result of the above-described determining unit.

[Item 2-12]

A picking system including

the control device according to any one of Items 2-1 to 2-11; and

the above-described picking apparatus.

[Item 2-13]

A distribution system including

the control device according to any one of Items 2-1 to 2-10;

the above-described picking apparatus; and

a conveying apparatus to convey the above-described work.

[Item 2-14]

A tangible, non-transitory computer-readable medium having processor instructions stored thereon that, when executed cause the processor to perform operations for controlling a picking apparatus that moves, by using a picking hand, one or more works from one or more supply containers to one or more arrangement containers, the operations comprising:

generating trajectory information that defines a trajectory for which the picking hand picks one of the one or more works stored in one supply container included in the one or more supply containers and arranges the work in one arrangement container included in the one or more arrangement containers, wherein

the generating trajectory information includes generating, after performing a process of generating trajectory information representing a first work included in the one or more works and before a movement of the first work is completed, trajectory information representing a second work, included in the one or more works and different from the first work, by using an operation result in the process of generating the trajectory information representing the first work.

[Item 2-15]

A control method for controlling a picking apparatus that arranges, on a second position of a second container, a work picked on a first position of a first container by using a picking hand, the second container being different from the above-described first container, the control method including:

a trajectory information generating step of generating trajectory information defining a trajectory for which the above-described picking hand picks the work on the above-described first position and arranges the work on the above-described second position;

an execution control step of operating the above-described picking apparatus based on the trajectory information generated in the above-described trajectory information generating step; and

an execution time estimating step of estimating a period of time that is from the time when the above-described picking apparatus receives an instruction for starting an operation on the above-described work in the above-described execution control step to the time when the operation of the above-described picking apparatus on the above-described work is ended, wherein

the above-described trajectory information generating step includes a step of adjusting the amount of calculation based on an estimation result in the above-described execution time estimating step.

[Item 2-16]

A production method including:

producing the one or more arrangement containers on which one or more works are arranged by performing the control method according to Item 2-15.

[Item 3-1]

A control device for controlling a picking apparatus that arranges, on a second position of a second container, a work picked on a first position of a first container by using a picking hand, the second container being different from the above-described first container, the control device including:

a point cloud information acquiring unit for acquiring point cloud information of a region including at least one of the above-described picking hand and the above-described work at a timing being at least one timing of (i) a timing at which the above-described picking hand picks the above-described work on the above-described first position, (ii) a timing at which the above-described picking hand moves the above-described work out of the above-described first container, (iii) a timing at which the above-described picking hand moves the above-described work in the above-described second container and (iv) a timing at which the above-described picking hand arranges the above-described work on the above-described second position; and

an abnormality detection unit for detecting an abnormality based on the point cloud information acquired by the above-described point cloud information acquiring unit.

[Item 3-2]

The control device according to Item 3-1, further including an image acquiring unit for acquiring image data of an image including at least one of the above-described picking hand and the above-described work in a field angle at the above-described at least one timing, wherein

the above-described point cloud information acquiring unit acquires the above-described point cloud information based on the image acquired by the above-described image acquiring unit.

[Item 3-3]

The control device according to Item 3-1 or 3-2, wherein

the above-described point cloud information is three-dimensional point cloud information.

[Item 3-4]

The control device according to any one of Items 3-1 to 3-3, wherein

the above-described abnormality detection unit detects an abnormality based on (i) the above-described point cloud information and (ii) at least one selected from a group including a measurement value of a sensor arranged on the above-described picking apparatus, a control value of at least one of the above-described picking hand and a robot arm coupled with the above-described picking hand, and at least one of torque, a voltage value and a current value of a driving unit that drives at least one of the above-described picking hand and the above-described robot arm.

[Item 3-5]

The control device according to any one of Items 3-1 to 3-4, further including an instruction sending unit for sending, to the above-described picking apparatus, an instruction for suspending an operation of the above-described picking apparatus when the above-described abnormality detection unit detects the abnormality.

[Item 3-6]

A picking system including

a control device according to any one of Items 3-1 to 3-5; and

the above-described picking apparatus.

[Item 3-7]

A distribution system including

the control device according to any one of Items 3-1 to 3-5;

the above-described picking apparatus; and

a conveying apparatus to convey the above-described work.

[Item 3-8]

A tangible, non-transitory computer-readable medium having processor instructions stored thereon that, when executed cause the processor to perform operations for controlling a picking apparatus that moves, by using a picking hand, one or more works from one or more supply containers to one or more arrangement containers, the operations comprising:

generating trajectory information that defines a trajectory for which the picking hand picks one of the one or more works stored in one supply container included in the one or more supply containers and arranges the work in one arrangement container included in the one or more arrangement containers, wherein

the generating trajectory information includes generating, after performing a process of generating trajectory information representing a first work included in the one or more works and before a movement of the first work is completed, trajectory information representing a second work, included in the one or more works and different from the first work, by using an operation result in the process of generating the trajectory information representing the first work.

[Item 3-9]

A control method for controlling a picking apparatus that arranges, on a second position of a second container, a work picked on a first position of a first container by using a picking hand, the second container being different from the above-described first container, the control method including:

a point cloud information acquiring procedure of acquiring point cloud information of a region including at least one of the above-described picking hand and the above-described work at a timing being at least one timing of (i) a timing at which the above-described picking hand picks the above-described work on the above-described first position, (ii) a timing at which the above-described picking hand moves the above-described work out of the above-described first container, (iii) a timing at which the above-described picking hand moves the above-described work in the above-described second container and (iv) a timing at which the above-described picking hand arranges the above-described work on the above-described second position; and

An abnormality detection procedure of detecting an abnormality based on the point cloud information acquired in the above-described point cloud information acquiring procedure.

[Item 4-1]

A control device for controlling a picking apparatus that moves one or more works from one or more supply containers to one or more arrangement containers by using a picking hand, the control device including

a trajectory information generating unit for generating trajectory information defining a trajectory for which the above-described picking hand picks one of the above-described one or more works stored in one supply container included in the above-described one or more supply containers and arranges the work in one arrangement container included in the above-described one or more arrangement containers, wherein

after the process of generating trajectory information representing a first work included in the above-described one or more work is performed and before the movement of the above-described first work is completed,

the above-described trajectory information generating unit generates trajectory information representing a second work being different from the above-described first work and included in the above-described one or more work by using an operation result in the process for generating the trajectory information representing the above-described first work.

[Item 4-2]

The control device according to Item 4-1, further including a container information acquiring unit for acquiring information representing at least one of a state of a surface of the above-described one supply container and a state of a surface of the above-described one arrangement container, wherein

the above-described trajectory information generating unit is for (a) acquiring or generating information representing a position and a posture of the above-described first work in the above-described one supply container before the above-described picking apparatus moves the above-described first work in processing for generating the trajectory information representing the above-described first work, and

(b) generating trajectory information representing the above-described second work by using (i) the information acquired by the above-described container information acquiring unit before the movement of the above-described first work is completed and (ii) the information representing the position and the posture of the above-described first work in the above-described one supply container during the process of generating the trajectory information representing the above-described second work.

[Item 4-3]

The control device according to Item 4-1, further including a container information acquiring unit for acquiring information representing a state of a surface of at least one of the above-described one supply container and the above-described one arrangement container, wherein

the above-described trajectory information generating unit is for (a) acquiring or generating information representing a position and a posture of the above-described first work in the above-described one arrangement container after the above-described picking apparatus moves the above-described first work during the process of generating the trajectory information representing the above-described first work, and

(b) generating trajectory information representing the above-described second work by using (i) information acquired by the above-described container information acquiring unit before the movement of the above-described first work is completed and (ii) information representing a position and a posture of the above-described first work in the above-described one arrangement container during the process of generating the trajectory information representing the above-described second work.

[Item 4-4]

The control device according to any one of Items 4-1 to 4-3, further including an order information acquiring unit for acquiring (i) first order information that includes instructions to move the above-described one or more works from at least one of the above-described one or more supply containers to a first arrangement container included in the above-described one or more arrangement containers and (ii) second order information that instruct to move the above-described one or more works from at least one of the above-described one or more supply containers to a second arrangement container that is included in the above-described one or more arrangement containers and different from the above-described first arrangement container, wherein

the above-described trajectory information generating unit is for performing a process of generating trajectory information representing the above-described first work in accordance with one of the above-described first order information and the above-described second order information acquired by the above-described order information acquiring unit, and

performing the process of generating trajectory information representing the above-described second work in accordance with the other of the above-described first order information and the above-described second order information acquired by the above-described order information acquiring unit.

[Item 4-5]

The control device according to Item 4-4, wherein

the above-described first order information includes instructions to move the above-described first work from a first supply container included in the above-described one or more supply containers to the above-described first arrangement container,

the above-described second order information includes instructions to move the above-described second work from the above-described first supply container to the above-described second arrangement container, and

the above-described trajectory information generating unit is for performing the process of generating trajectory information representing the above-described first work in accordance with the above-described first order information acquired by the above-described order information acquiring unit, and

performing the process of generating the trajectory information representing the above-described second work in accordance with the above-described second order information acquired by the above-described order information acquiring unit.

[Item 4-6]

The control device according to any one of Items 4-1 to 4-3, further including an order information acquiring unit for acquiring (i) first order information that includes instructions to move the above-described one or more works from a first supply container included in the above-described one or more supply containers to at least one of the above-described one or more arrangement containers and (ii) second order information that includes instructions to move the above-described one or more works from a second supply container, different from the first supply container and included in the above-described one or more supply containers and different from the above-described first supply container, to at least one of the above-described one or more arrangement containers, wherein

the above-described trajectory information generating unit is for performing the process of generating trajectory information representing the above-described first work in accordance with one of the above-described first order information and the above-described second order information acquired by the above-described order information acquiring unit, and

-   -   performing the process of generating trajectory information         representing the above-described second work in accordance with         the other of the above-described first order information and the         above-described second order information acquired by the         above-described order information acquiring unit.         [Item 4-7]

The control device according to Item 4-6, wherein

the above-described first order information includes instructions to move the above-described first work from the above-described first supply container to a first arrangement container included in the above-described one or more supply containers,

the above-described second order information includes instructions to move the above-described second work from the above-described second supply container to the above-described first arrangement container, and

the above-described trajectory information generating unit is for performing the process of generating the trajectory information representing the above-described first work in accordance with the above-described first order information acquired by the above-described order information acquiring unit, and

performing the process of generating trajectory information representing the above-described second work in accordance with the above-described second order information acquired by the above-described order information acquiring unit.

[Item 4-8]

The control device according to any one of Items 4-1 to 4-3, further including an order information acquiring unit for acquiring (i) first order information that includes instructions to move the above-described first work from first supply container included in the above-described one or more supply containers to a first arrangement container included in the above-described one or more arrangement containers and (ii) second order information that includes instructions to move the above-described second work from a second supply container, different from the first supply container and included in the above-described one or more supply containers and different from the above-described first supply container, to a second arrangement container that is included in the above-described one or more arrangement containers and different from the above-described first arrangement container, and

the above-described trajectory information generating unit is for performing the process of generating trajectory information representing the above-described first work in accordance with the above-described first order information acquired by the above-described order information acquiring unit, and performs the process of generating the trajectory information representing the above-described second work in accordance with the above-described second order information acquired by the above-described order information acquiring unit.

[Item 4-9]

A picking system including

the control device according to any one of Items 4-1 to 4-8; and

the above-described picking apparatus.

[Item 4-10]

A distribution system including

the control device according to any one of Items 4-1 to 4-8;

the above-described picking apparatus; and

a conveying apparatus to convey the above-described one or more works.

[Item 4-11]

A tangible, non-transitory computer-readable medium having processor instructions stored thereon that, when executed cause the processor to perform operations for controlling a picking apparatus that moves, by using a picking hand, one or more works from one or more supply containers to one or more arrangement containers, the operations comprising:

generating trajectory information that defines a trajectory for which the picking hand picks one of the one or more works stored in one supply container included in the one or more supply containers and arranges the work in one arrangement container included in the one or more arrangement containers, wherein

the generating trajectory information includes generating, after performing a process of generating trajectory information representing a first work included in the one or more works and before a movement of the first work is completed, trajectory information representing a second work, included in the one or more works and different from the first work, by using an operation result in the process of generating the trajectory information representing the first work.

[Item 4-12]

A control method for controlling a picking apparatus that moves one or more works from one or more supply containers to one or more arrangement containers by using a picking hand, the control method including

a trajectory information generating step of generating trajectory information defining a trajectory for which the above-described picking hand picks one of the above-described one or more works stored in one supply container included in the above-described one or more supply containers and arranges the work in one arrangement container included in the above-described one or more arrangement containers, wherein

the above-described trajectory information generating step includes

a step of generating trajectory information representing a second work, included in the above-described one or more works and different from the above-described first work, by using ab operation result in the process of generating the trajectory information representing the above-described first work after performing the process of generating the trajectory information representing a first work included in the above-described one or more works and before the movement of the above-described first work is completed.

[Item 4-13]

A production method including:

producing the one or more arrangement containers on which one or more works are arranged by performing the control method according to Item 4-12.

[Item 5-1]

An information processing apparatus including:

a three-dimensional information acquiring unit for acquiring three-dimensional information of a target object;

a holding information acquiring unit for acquiring holding information that defines a method of holding the above-described target object by using a picking hand; and

an information management unit for storing, in a storage apparatus, (i) three-dimensional information of the above-described target object acquired by the above-described three-dimensional information acquiring unit and (ii) the holding information acquired by the above-described holding information acquiring unit in association with each other.

[Item 5-2]

The information processing apparatus according to Item 5-1, wherein

the three-dimensional information of the above-described target object includes at least one of home position information and coordinate axis information.

[Item 5-3]

The information processing apparatus according to Item 5-1 or 5-2, further including a two-dimensional information acquiring unit for acquiring two-dimensional information of the above-described target object, wherein

the above-described information management unit stores, in the above-described storage apparatus, (i) three-dimensional information of the above-described target object acquired by the above-described three-dimensional information acquiring unit, (ii) holding information acquired by the above-described holding information acquiring unit and (iii) two-dimensional information of the above-described target object acquired by the above-described two-dimensional information acquiring unit in association with each other.

[Item 5-4]

The information processing apparatus according to Item 5-3, further including a feature information acquiring unit for acquiring feature information for recognizing the above-described target object based on the two-dimensional information of the above-described target object acquired by the above-described two-dimensional information acquiring unit, wherein

the above-described information management unit stores, in the above-described storage apparatus, (i) three-dimensional information of the above-described target object acquired by the above-described three-dimensional information acquiring unit, (ii) holding information acquired by the above-described holding information acquiring unit, (iii) two-dimensional information of the above-described target object acquired by the above-described two-dimensional information acquiring unit and (iv) feature information of the above-described target object acquired by the above-described feature information acquiring unit in association with each other.

[Item 5-5]

The information processing apparatus according to Item 5-1 or 5-2, further including:

a two-dimensional information acquiring unit for acquiring two-dimensional information of the above-described target object; and

a feature information acquiring unit for acquiring feature information for recognizing the above-described target object based on two-dimensional information of the above-described target object acquired by the above-described two-dimensional information acquiring unit, wherein

the above-described information management unit stores, in the above-described storage apparatus, (i) three-dimensional information of the above-described target object acquired by the above-described three-dimensional information acquiring unit, (ii) holding information acquired by the above-described holding information acquiring unit and (iii) feature information of the above-described target object acquired by the above-described feature information acquiring unit in association with each other.

[Item 5-6]

The information processing apparatus according to any one of Items 5-3 to 5-5, wherein

the above-described two-dimensional information is image data of three or more images of the above-described target object captured from three or more directions.

[Item 5-7]

The information processing apparatus according to any one of Items 5-1 to 5-6, further including the above-described storage apparatus.

[Item 5-8]

The information processing apparatus according to any one of Items 5-1 to 5-7, wherein

the above-described information management unit stores, in a storage apparatus, (i) three-dimensional information of the above-described target object acquired by the above-described three-dimensional information acquiring unit, (ii) holding information acquired by the above-described holding information acquiring unit and (iii) identification information for identifying the above-described target object in association with each other.

[Item 5-9]

The information processing apparatus according to Item 5-8, further including:

a picking control device to control a picking apparatus that arranges, on a second position, a work picked on a first position by using a picking hand, the second position being different from the above-described first position;

an order information acquiring unit for acquiring order information including identification information of at least one work that is to be a working target of the picking apparatus;

a holding information storing unit for storing identification information of one or more works and holding information of the above-described one or more works in association with each other; and

a holding information extracting unit for (i) accessing to the above-described storage apparatus and extract, from identification information of the above-described at least one work included in order information acquired by the above-described order information acquiring unit, the above-described holding information associated with identification information of the above-described target object matched with identification information that is not stored in the above-described holding information storing unit, and (ii) associating the identification information that is not stored in the above-described holding information storing unit with the above-described extracted holding information and store the information in the above-described holding information storing unit, wherein

the above-described holding information storing unit is realized by a medium being different from the above-described storage apparatus.

[Item 5-10]

The information processing apparatus according to Item 5-9, further including:

a first candidate information generating unit for generating first candidate information indicating each of one or more postures that the above-described picking hand takes on the above-described first position with reference to the holding information stored in the above-described holding information storing unit;

a second candidate information generating unit for generating second candidate information indicating each of one or more postures that the above-described picking hand takes on the above-described second position with reference to the holding information stored in the above-described holding information storing unit;

an extracting unit for extracting one combination from among one or more combinations regarding one or more postures indicated by the above-described first candidate information and one or more postures indicated by the above-described second candidate information;

a trajectory calculating unit for calculating a trajectory for which (i) the above-described picking hand picks, on the above-described first position, the above-described work with a posture associated with the above-described first position included in the above-described one combination and (ii) the above-described picking hand arranges, on the above-described second position, the above-described work with a posture associated with the above-described second position included in the above-described one combination; and

a determining unit for determining whether an interference is present or absent on the trajectory calculated by the above-described trajectory calculating unit.

[Item 5-11]

A picking system including

the information processing apparatus according to Item 5-9 or 5-10; and

the above-described picking apparatus.

[Item 5-12]

A distribution system including

the information processing apparatus according to Item 5-9 or 5-10;

the above-described picking apparatus; and

a conveying apparatus to convey the above-described work.

[Item 5-13]

A tangible, non-transitory computer-readable medium having processor instructions stored thereon that, when executed cause the processor to perform operations for controlling a picking apparatus that moves, by using a picking hand, one or more works from one or more supply containers to one or more arrangement containers, the operations comprising:

generating trajectory information that defines a trajectory for which the picking hand picks one of the one or more works stored in one supply container included in the one or more supply containers and arranges the work in one arrangement container included in the one or more arrangement containers, wherein

the generating trajectory information includes generating, after performing a process of generating trajectory information representing a first work included in the one or more works and before a movement of the first work is completed, trajectory information representing a second work, included in the one or more works and different from the first work, by using an operation result in the process of generating the trajectory information representing the first work.

[Item 5-14]

An information processing method including:

a three-dimensional information acquiring step of acquiring, by a computer, three-dimensional information of a target object;

a holding information acquiring step of acquiring holding information that defines a method of holding, by a computer, the above-described target object by using a picking hand; and

an information management step of storing, in a storage apparatus by a computer, (i) three-dimensional information of the above-described target object acquired by the above-described three-dimensional information acquiring unit and (ii) holding information acquired by the above-described holding information acquiring unit in association with each other.

EXPLANATION OF REFERENCES

10 . . . work; 11 . . . supply pallet; 12 . . . supply pallet; 14 . . . delivery pallet; 15 . . . delivery pallet; 100 . . . distribution management system; 110 . . . article management apparatus; 120 . . . data registration apparatus; 130 . . . transfer apparatus; 140 . . . robot; 151 . . . image capturing apparatus; 152 . . . image capturing apparatus; 154 . . . image capturing apparatus; 155 . . . image capturing apparatus; 160 . . . transfer control apparatus; 170 . . . conveying apparatus; 171 . . . conveyor; 172 . . . conveyor; 174 . . . conveyor; 175 . . . conveyor; 210 . . . order information acquiring unit; 220 . . . conveying apparatus management unit; 230 . . . transfer apparatus management unit; 240 . . . request processing unit; 250 . . . article information storing unit; 312 . . . three-dimensional information acquiring unit; 314 . . . two-dimensional information acquiring unit; 316 . . . mass information acquiring unit; 320 . . . article information registration unit; 330 . . . input/output unit; 340 . . . registration information generating unit; 342 . . . coordinate information adding unit; 344 . . . holding information adding unit; 346 . . . specification information adding unit; 350 . . . registration request sending unit; 410 . . . manipulator; 412 . . . picking hand; 414 . . . robot arm; 420 . . . driving unit; 430 . . . drive control unit; 510 . . . recognition information storing unit; 520 . . . recognition unit; 530 . . . planning unit; 540 . . . execution unit; 550 . . . concurrent processing control unit; 610 . . . position and posture information generating unit; 620 . . . point cloud information generating unit; 710 . . . holding planning unit; 720 . . . arrangement planning unit; 730 . . . evaluation information storing unit; 740 . . . trajectory candidate extracting unit; 750 . . . trajectory planning unit; 760 . . . control information generating unit; 810 . . . control information sending unit; 820 . . . execution monitoring unit; 1400 . . . data table; 1410 . . . work ID; 1420 . . . order ID; 1430 . . . contents; 1432 . . . article ID; 1434 . . . supply pallet ID; 1436 . . . delivery pallet ID 

What is claimed is:
 1. A control device for controlling a picking apparatus that moves, by using a picking hand, one or more works from one or more supply containers to one or more arrangement containers, the control device comprising: a trajectory information generating unit for generating trajectory information defining a trajectory for which the picking hand picks one of the one or more works stored in one supply container included in the one or more supply containers and arranges the work in one arrangement container included in the one or more arrangement containers; a container information acquiring unit for acquiring information representing a state of a surface of at least one of the one supply container and the one arrangement container, wherein after performing a process of generating trajectory information representing a first work included in the one or more works and before a movement of the first work is completed, the trajectory information generating unit generates trajectory information representing a second work, included in the one or more works and different from the first work, by reusing information obtained or generated in the process of generating the trajectory information representing the first work, the information obtained or generated in the process of generating the trajectory information representing the first work includes at least one of: the information representing the state of the surface of the at least one of the one supply container and the one arrangement container obtained by the container information acquiring unit; information representing a position and a posture that the picking hand is operable to pick the first work included in the trajectory information generated by the trajectory information generating unit; and information representing a position and a posture that the picking hand is operable to arrange the first work included in the trajectory information generated by the trajectory information generating unit, the trajectory information generating unit, in a process of generating trajectory information representing the second work: (a) in a case that the first work and the second work are stored in the one supply container, determines a position and a posture that the picking hand is operable to pick the second work or determines whether an interference is present or absent on a trajectory candidate of the second work based on (i) information acquired by the container information acquiring unit before a movement of the first work is completed and (ii) information representing a position and a posture that the picking hand is operable to pick the first work obtained or generated in the process of generating the trajectory information representing the first work, or (b) in a case that the first work and the second work are arranged in the one arrangement container determines a position and a posture that the picking hand is operable to arrange the second work or determines whether an interference is present or absent on a trajectory candidate of the second work based on (i) information acquired by the container information acquiring unit before a movement of the first work is completed and (ii) information representing a position and a posture that the picking hand is operable to arrange the first work obtained or generated in the process of generating the trajectory information representing the first work.
 2. The control device according to claim 1, wherein the trajectory information generating unit is for: (a) acquiring or generating information representing a position and a posture of the first work in the one supply container before the picking apparatus moves the first work during the process of generating the trajectory information representing the first work; and (b) generating trajectory information representing the second work by using (i) information acquired by the container information acquiring unit before the movement of the first work is completed and (ii) information representing the position and the posture of the first work in the one supply container during the process of generating the trajectory information regarding the second work.
 3. The control device according to claim 1, wherein the trajectory information generating unit is for: (a) acquiring or generating information representing a position and a posture of the first work in the one arrangement container after the picking apparatus moves the first work during the process of generating the trajectory information regarding the first work; and (b) generating trajectory information representing the second work by using (i) information acquired by the container information acquiring unit before the movement of the first work is completed and (ii) the information representing the position and the posture of the first work in the one arrangement container during the process of generating the trajectory information representing the second work.
 4. The control device according to claim 1, further comprising an order information acquiring unit for acquiring: (i) first order information that includes instructions to move the first work from a first supply container included in the one or more supply containers to a first arrangement container included in the one or more arrangement containers and (ii) second order information that includes instructions to move the second work from a second supply container, that is different from the first supply container and included in the one or more supply containers and different from the first supply container, to a second arrangement container that is included in the one or more arrangement containers and different from the first arrangement container, wherein the trajectory information generating unit performs: a process of generating trajectory information representing the first work in accordance with the first order information acquired by the order information acquiring unit, and a process of generating trajectory information representing the second work in accordance with the second order information acquired by the order information acquiring unit.
 5. The control device according to claim 1, wherein the trajectory information generating unit, in the process of generating trajectory information representing the second work: (a) in the case that the first work and the second work are stored in the one supply container, determines the position and the posture that the picking hand is operable to pick the second work based on (i) the information acquired by the container information acquiring unit before the movement of the first work is completed and (ii) the information representing the position and the posture that the picking hand is operable to pick the first work obtained or generated in the process of generating the trajectory information representing the first work, or (b) in the case that the first work and the second work are arranged in the one arrangement container determines the position and the posture that the picking hand is operable to arrange the second work based on (i) the information acquired by the container information acquiring unit before the movement of the first work is completed and (ii) the information representing the position and the posture that the picking hand is operable to arrange the first work obtained or generated in the process of generating the trajectory information representing the first work.
 6. The control device according to claim 1, further comprising an execution sequence determining unit for (i) acquiring information representing a plurality of works that is to be a working target of the picking apparatus, and (ii) determining an execution sequence of the plurality of works such that a calculation amount or a calculation time by the trajectory information generating unit is reduced or such that a working time or a waiting time of the picking apparatus is reduced, wherein the trajectory information generating unit generates the trajectory information representing the first work and the trajectory information representing the second work according to the execution sequence determined by the execution sequence determining unit.
 7. A tangible, non-transitory computer-readable medium having processor instructions stored thereon that, when executed cause the processor to perform operations for controlling a picking apparatus that moves, by using a picking hand, one or more works from one or more supply containers to one or more arrangement containers, the operations comprising: generating trajectory information that defines a trajectory for which the picking hand picks one of the one or more works stored in one supply container included in the one or more supply containers and arranges the work in one arrangement container included in the one or more arrangement containers; and acquiring information representing a state of a surface of at least one of the one supply container and the one arrangement container, wherein the generating trajectory information includes generating, after performing a process of generating trajectory information representing a first work included in the one or more works and before a movement of the first work is completed, trajectory information representing a second work, included in the one or more works and different from the first work, by reusing information obtained or generated in the process of generating the trajectory information representing the first work, the information obtained or generated in the process of generating the trajectory information representing the first work includes at least one of: the information representing the state of the surface of the at least one of the one supply container and the one arrangement container obtained in the acquiring information representing the state of the surface; information representing a position and a posture that the picking hand is operable to pick the first work included in the trajectory information generated in the generating trajectory information; and information representing a position and a posture that the picking hand is operable to arrange the first work included in the trajectory information generated in the generating trajectory information, the generating trajectory information includes: in a process of generating trajectory information representing the second work: (a) in a case that the first work and the second work are stored in the one supply container, determining a position and a posture that the picking hand is operable to pick the second work or determines whether an interference is present or absent on a trajectory candidate of the second work based on (i) information acquired in the acquiring information representing the state of the surface before a movement of the first work is completed and (ii) information representing a position and a posture that the picking hand is operable to pick the first work obtained or generated in the process of generating the trajectory information representing the first work, or (b) in a case that the first work and the second work are arranged in the one arrangement container determining a position and a posture that the picking hand is operable to arrange the second work or determines whether an interference is present or absent on a trajectory candidate of the second work based on (i) information acquired in the acquiring information representing the state of the surface before a movement of the first work is completed and (ii) information representing a position and a posture that the picking hand is operable to arrange the first work obtained or generated in the process of generating the trajectory information representing the first work.
 8. A control method for controlling a picking apparatus that moves, by using a picking hand, one or more works from one or more supply containers to one or more arrangement containers, the control method comprising: generating trajectory information that defines a trajectory for which the picking hand picks one of the one or more works stored in one supply container included in the one or more supply containers and arranges the work in one arrangement container included in the one or more arrangement containers; and acquiring information representing a state of a surface of at least one of the one supply container and the one arrangement container, wherein the generating trajectory information includes generating, after performing a process of generating trajectory information representing a first work included in the one or more works and before a movement of the first work is completed, trajectory information regarding a second work, included in the one or more works and different from the first work, by reusing information obtained or generated in the process of generating the trajectory information regarding the first work, the information obtained or generated in the process of generating the trajectory information representing the first work includes at least one of: the information representing the state of the surface of the at least one of the one supply container and the one arrangement container obtained in the acquiring information representing the state of the surface; information representing a position and a posture that the picking hand is operable to pick the first work included in the trajectory information generated in the generating trajectory information; and information representing a position and a posture that the picking hand is operable to arrange the first work included in the trajectory information generated in the generating trajectory information, the generating trajectory information includes: in a process of generating trajectory information representing the second work: (a) in a case that the first work and the second work are stored in the one supply container, determining a position and a posture that the picking hand is operable to pick the second work or determines whether an interference is present or absent on a trajectory candidate of the second work based on (i) information acquired in the acquiring information representing the state of the surface before a movement of the first work is completed and (ii) information representing a position and a posture that the picking hand is operable to pick the first work obtained or generated in the process of generating the trajectory information representing the first work, or (b) in a case that the first work and the second work are arranged in the one arrangement container determining a position and a posture that the picking hand is operable to arrange the second work or determines whether an interference is present or absent on a trajectory candidate of the second work based on (i) information acquired in the acquiring information representing the state of the surface before a movement of the first work is completed and (ii) information representing a position and a posture that the picking hand is operable to arrange the first work obtained or generated in the process of generating the trajectory information representing the first work. 